As it turns out, my Web hosting service, WestHost, had suffered a significant outage that took out Lionel Deimel’s Farrago and Lionel Deimel’s Web Log, not to mention other sites for which I am responsible. WestHost kept moving into the future the time when the problem was supposed to be fixed. Not until this afternoon did the company offer an explanation:
On Saturday, February 20 2010, at approximately 2:20 p.m. MST during an annual fire system inspection of our data center the fire suppression system was inadvertently triggered and discharged into the data center environment. The discharge of the fire system has affected the hardware and operations of our servers. Not all servers have been affected by the discharge. We are working to restore servers and services as quickly as possible. Due to the nature of this interruption we will be experiencing extended outages.This explanation was posted nearly 24 hours after the incident. Before that, WestHost had provided only vague explanations about what had happened. Did it really take these folks 24 hours to figure out that their servers were covered with fire-suppressant foam or were they trying to avoid admitting what had happened?
Anyway, I am happy to report that my Web site, blog, and e-mail all seem to be working again. I apologize for any inconvenience, and sincerely hope this problem will not recur.
I should say that I am a WestHost client for good reason, and what happened yesterday was an extraordinary event. Even with the ongoing crisis, I was able to speak to a tech support person whose native language was clearly American English without too long a wait on the telephone. At the moment, however, I am not a happy camper.
Update (2/22/2010, 8:05 AM ET): Although my blog and Web site are on the Web today, my e-mail and ftp service are not working. WestHost estimates a fix will take another two hours.
Update (2/22/2010, 1:36 PM ET): WestHost is now estimating another 8-1/2 hours to a fix. Still no e-mail.
Update (2/23/2010, 7:34 AM ET): When I went to bed last night, the status of my hosting account was unchanged, but everything seems to be working this morning. WestHost has provided more information about the failure and their response to it. Some servers may be down until Friday. Apparently, a contractor was responsible for triggering the fire suppression system.