Thursday, June 07, 2007

Power went off

In early to try to cut down the mountain of projects and I crack on with the first on my pile. About 30 minutes into it and my PC loses its network connection. Lose cable? Nope.

Up in the server room and I find one Cisco 3750 fabric switch complaining miserably. Assuming that the RPS or the switch itself have got a problem, the switch gets restarted. Two goes and still it isn't happy. Then we see some servers are running and not others.

One UPS has gone to battery, run out of juice and taken all equipment down with it. Naturally this includes our Exchange server, its RAID array, another RAID array and 2 other servers.

So someone tell me why when we bring email up do I find the "Stores" folder on the RAID array is completely missing? 200GB of emails all gone on holiday. Yay for a full backup and transaction logs stored on internal disks. Set stores to overwrite and hit restore.

The new server has a faster tape drive, much faster disks, a lot more RAM and hopefully a MUCH better RAID array that doesn't randomly throw out 200GB of email.

Why did the UPS go to battery? I can't prove anything so I shall not be casting nastertiums.

No comments: