Thursday, April 19, 2007

ccM is down

ccM has been down for about 8-9 hours. We're waiting for the bots to pass so we can safely get back up and running.


b.kommerij said...
This comment has been removed by the author.
teru said...

That was a long one. What happened?

fourstones said...

Google allows webmasters to control the 'speed' (or more accurately 'ferocity') with which it crawls your site. For some reason that setting (which I set to 'slow') expires on it's own and resets itself to 'normal' which is more than our servers can handle.

Our configuration database was corrupted a few minutes after the first 'normal' crawl started. 16 hours laters Google bot was STILL banging away acting surprisingly like a very ill behaved spam bot (hitting the same pages multiple times, ignoring 100,000s 'access denied' messages we threw at it and ignoring our 'do not search' insructions they claim to honor).

I was finally able to isolate them and work on the actual corruption problem.

The drag is we run the risk of being "de listed" on google which would be really bad site - almost as bad as being taken down by their abuse.

teru said...

damn. that sucks.

I didn't realize the server was so maxed out. Also didn't realize Google was so aggressive. Sounds like a trojan virus.

I'm glad it got sorted.