PDA

View Full Version : Bandwidth Consumption **UPDATED**



Sherry
10-27-2008, 09:09 AM
Hi gals,

I am once again beginning to receive email messages from our server that we are at 85% of our alloted bandwidth consumption. Once we reach 100% this means that our site will go offline until the beginning of the next month.

We've been running smoothly for over two years with no problems and, all of a sudden, have been experiencing website time outs and huge bandwidth consumption. I am not sure why this is happening but I will be investigating the source of the problem and, if necessary, will make changes to alleviate the strain on our server.

I want to remind everyone that Shuggy has set up an emergency Yahoo group so that if our website does go offline you can stay informed about what is taking place. You can find the website here: http://groups.yahoo.com/group/shackieupdates/

I'll keep you all updated on this issue.

Sherry


************************************************** ************************************************** ******************************
***UPDATE***
************************************************** ************************************************** ******************************

Hi Ladies,

I have spent most of the day investigating the source of our bandwidth consumption. The good news is that it isn't anything our Shack members are doing (ie. uploading photos to the gallery or adding images to your posts, or having multiple PMs stored).

Most of our bandwidth, believe it or not, is being used up by spiders, robots, and crawlers. What on earth is that you ask? Well, A spider, robot, or crawler is a program that follows, or "crawls", links throughout the internet, grabbing content from sites and adding it to search engine indexes. Yahoo is the biggest culprit and is sending out hundreds of spiders to our website at one time. They have hit our website 142,742 times in the past month and have used up 5.57 gig of our bandwidth! Second in line is Google and they have used up 3.57 gig.

Along with the pesky spiders, guests who visit The Stamp Shack website are also using up bandwidth. They spend time browsing our gallery and reading the messages on the board but don't care to register for an account. We sometimes have 50-100 guests visiting at one time so, as you can imagine, this is eating up a tremendous amount of bandwidth.

In light of this discovery, I have implemented a few changes to the website.

1. I have installed a program that will limit the amount of spiders/bots crawling our website AND will limit what links they actually grab. Unfortunately, it can take up to three weeks before this takes effect.

2. Guests are no longer permitted to view the content of our website. If they wish to view the gallery or the message board, they must register for a basic account. I really hate doing this but I cannot allow our website to go offline each and every month in order to accomodate guest viewing.

3. For this month only, I have disabled a few non-essential features (such as profile messages, member albums, and the friends list) in hopes that we can make it through the rest of the month without going offline. These features will be restored on November 1st.

So many people have emailed me today with suggestions on how we might conserve bandwidth and I'd like to thank each and every one of you for your concern. You're all so sweet! Some people have suggested that we have our bandwidth limit increased but, as of right now, I'm not willing to do that. We are currently forking over $150 per month for our dedicated server and I really am not comfortable paying any more than that.

Please go about your normal Shack activities. I think that by initiating the changes outlined above we should see a decrease in bandwidth consumption over the next few weeks. Of course, it doesn't help us out this month as we are already at 90% of our quota. Remember, if we do go offline, please check in with Angie's Yahoo Group for information and updates.

Thanks so much for your concern and your understanding!! You really are a great bunch of gals :)

mahodgman
10-27-2008, 09:48 AM
Is there anything we can do to help, Sherry? I'm not sure what activities put a strain on the bandwidth, but if you could tell us I'd be happy to stop uploading (for example) for a week or something to help out.

logcabincreations
10-27-2008, 10:05 AM
that really is odd....do you think a hacker might be tampering with the site or something (sorry I don't know much about these things.)

cindi
10-27-2008, 10:37 AM
is there anything we can do to help?? please let us know, I'm sure we would work togetherr.
:wideeyed:

Mrs. Fence
10-27-2008, 10:46 AM
:confused:I'm with the others - low on tech skills and knowlege - big on doing what I can to help!

ccstampin12
10-27-2008, 11:06 AM
Geez, Sherry!
Can we make it for 4 and a half more days?
We can chill for the time being, not upload or whatever could help.
Let us know what would be helpful if you get it figured out!!!
Joanie

Nancy
10-27-2008, 11:13 AM
Does it have anything to do with the large pictures that are on different threads? - Nancy

Robinkay
10-27-2008, 11:41 AM
What can we do to help? How frustrating for you Sherry, if there is anything please let us know.

Monkey Love
10-27-2008, 02:27 PM
I am with the others! I deleted as many pm's as I could hoping this would help... Is there anything else?

Paula Carden
10-27-2008, 05:26 PM
I am wondering the same things- is it the message storage, pics etc... can we do anything different- maybe only so many pics per week to the gallery? Deleting PM's?

workin4stamps
10-27-2008, 05:31 PM
I could be wrong, and hopefully Sherry will respond to us, but I thought before she said it wasn't really the pm's and photos that took up so much bandwidth. I thought she said the majority of it was from template downloading. I'm not really sure though. If I need to remove my gigantic picture from my signature line, I surely will!!

Sherry
10-27-2008, 06:45 PM
Hi Ladies,

I have spent most of the day investigating the source of our bandwidth consumption. The good news is that it isn't anything our Shack members are doing (ie. uploading photos to the gallery or adding images to your posts, or having multiple PMs stored).

Most of our bandwidth, believe it or not, is being used up by spiders, robots, and crawlers. What on earth is that you ask? Well, A spider, robot, or crawler is a program that follows, or "crawls", links throughout the internet, grabbing content from sites and adding it to search engine indexes. Yahoo is the biggest culprit and is sending out hundreds of spiders to our website at one time. They have hit our website 142,742 times in the past month and have used up 5.57 gig of our bandwidth! Second in line is Google and they have used up 3.57 gig.

Along with the pesky spiders, guests who visit The Stamp Shack website are also using up bandwidth. They spend time browsing our gallery and reading the messages on the board but don't care to register for an account. We sometimes have 50-100 guests visiting at one time so, as you can imagine, this is eating up a tremendous amount of bandwidth.

In light of this discovery, I have implemented a few changes to the website.

1. I have installed a program that will limit the amount of spiders/bots crawling our website AND will limit what links they actually grab. Unfortunately, it can take up to three weeks before this takes effect.

2. Guests are no longer permitted to view the content of our website. If they wish to view the gallery or the message board, they must register for a basic account. I really hate doing this but I cannot allow our website to go offline each and every month in order to accomodate guest viewing.

3. For this month only, I have disabled a few non-essential features (such as profile messages, member albums, and the friends list) in hopes that we can make it through the rest of the month without going offline. These features will be restored on November 1st.

So many people have emailed me today with suggestions on how we might conserve bandwidth and I'd like to thank each and every one of you for your concern. You're all so sweet! Some people have suggested that we have our bandwidth limit increased but, as of right now, I'm not willing to do that. We are currently forking over $150 per month for our dedicated server and I really am not comfortable paying any more than that.

Please go about your normal Shack activities. I think that by initiating the changes outlined above we should see a decrease in bandwidth consumption over the next few weeks. Of course, it doesn't help us out this month as we are already at 90% of our quota. Remember, if we do go offline, please check in with Angie's Yahoo Group for information and updates.

Thanks so much for your concern and your understanding!! You really are a great bunch of gals :)

maxiesmom
10-27-2008, 06:49 PM
Spiders and robots and crawlers, oh my! Can't even imagine all this stuff, so beyond my scope.

shadoob
10-27-2008, 07:03 PM
Thank you, Sherry. I HATE spiders and crawly things anyway. I think your decision about guests is a realistic and important one.

cindi
10-27-2008, 07:05 PM
spiders, robots, creepy crawleys..sounds like Halloween!
Thanks Sherry for your time & effort in figuring this out. who would have thought it?
I wonder if we can make it to the end of the month. we will see
thanks for taking such good care of us Sherry!
:196:

Christie
10-27-2008, 07:06 PM
Sherry you are doing a great job!!! I know it stinks to have to implent changes to the site but I think it's the best thing right now. I love the shack and don't want to see us have any down time. Thanks for all of your hardwork!!!!

Mrs. Fence
10-27-2008, 07:23 PM
Thanks, Sherri. I'm sure that took some time and thought to figure out and correct the situation. I'm Glad we aren't going to be limited in our Shack activities. It makes sense to limit those who aren't members. I've really enjoyed my time here and look forward to lots more!

directmailscrapper
10-27-2008, 07:34 PM
Boy are we lucky to have you, Sherry! You understand all of this stuff that I don't have a clue about! Thanks for taking such good care of us!

ccstampin12
10-27-2008, 08:47 PM
I have NO idea what you just told us about the creepy crawlies in the site! I'm sure happy you understand all of this and know what to do about it!
I agree...you are doing a wonderful job and we all appreciate you and your continuing efforts on our behalf!
HUGS
Joanie

Canadian, eh
10-27-2008, 09:40 PM
Wow, that's quite something. Lots of work for you, unfortunately. I'll bet you never dreamed your site would get so popular. I'm so glad you started the Shack. Thanks for all your hard work.

mom_on_line
10-27-2008, 09:42 PM
Wow!

I think it is a good idea to have guests register to look at the site. There are many other boards that have the same requirements, so I suppose this is also the reason why.

I think implementing that would certainly cut the loss of the bandwidth by quite a bit.

hbpuppy
10-27-2008, 09:50 PM
Thanks for the update Sherry. I think your ideas sound good. I don't like the idea of spiders crawling all over us.

iloveflipflops
10-27-2008, 10:06 PM
Yeah Sherry - I don't like any kind of spiders. LOL

Jovi
10-28-2008, 11:11 AM
Yeah Sherry - I don't like any kind of spiders. LOL

yucky spiders...

Thanks Sherry :)

logcabincreations
10-28-2008, 12:41 PM
stupid robots...icky spiders....and I absolutely hate crawlers of any kind...lol

spredbirds
10-28-2008, 01:40 PM
thanks Sherri for looking out for us ... we will support anything that you need to keep us up and running.

Thanks again for all that you do.

janett
03-24-2010, 01:46 AM
you could store the picks on photobucket that way no bw from here
jan

Created By You
09-25-2011, 10:02 AM
Wow.. the changes you've implemented are all good, hopefully everything will be back to normal soon!

Stampo
09-25-2011, 03:55 PM
I had to shut down my Stamp Art Forum as we were being spammed mercilessly. Now on my VLVS! Forum, I get about 400 attempts a day but the confirmation email bounces... still a nuisance as I get the bounces... I show about 60,000 member of my forum which is absurd as there are really only a few active members for the swaps and games. I wish I knew how to counteract them. I get about one or two spammers a day and as soon as I see the posts, I delete them and report them to the AntiSpam thingie but doubt they really do anything about it.
8979