Page 1 of 1

Forum Downtime

Posted: Tue Jun 03, 2008 6:10 am
by Jorg
Hi All,

My apologies for the downtime of the forum. It appeared there was a big coincidence that the downtime of the forum happened on the same period as an explosion at a server data store. I am not 100% sure if this is where Jason keeps his servers, but it is very likely the downtime for days was related to this event. More info about it:

http://www.allheadlinenews.com/articles/7011131199

With regards,
- Jorgen

Posted: Wed Jun 04, 2008 6:37 am
by Sof_T
We are just glad to see you back :D

Posted: Wed Jun 04, 2008 7:57 am
by Jorg
As I was able to browse some of the forum yesterday morning, in the afternoon it was gone again. It seems there is some kind of ripple effect. Let's hope this was the last downtime. I'm glad to be back it is kind of your daily habit to check posts and reply to questions ;-)

- Jorgen

Posted: Wed Jun 04, 2008 9:38 am
by tierra
Quick Update:

Based on the behavior of the downtime on the forums (and the internet route to them), I think I can safely say that the forums are hosted at The Planet's Houston data center (H1) in Phase 1 (the first floor). The first level, containing 3,000 servers, is where the electrical equipment room that caught fire is located. They were able to bring the second floor back online with their own onsite generator within about 32 hours of the explosion powered, but the temporary generator delivered to power the first floor failed after a few hours of service (I think this is the outage that Jorg saw) and a second generator was needed to bring power back up again.

If the forums are in fact on a server in Phase 1 and not in Phase 2, we will see approximately 4-6 hours of downtime either on the 7th or the 14th at midnight (CDT I think) when they switch power back to the replacement battery backup system. We won't know until Jason can get back to us (he may not know himself if Idaho Web Hoster hasn't said). It's still possible that the second outage was caused by other DNS or network issues which H1 has been resolving while bringing up power to each floor, in which case, it's still possible that the forums are in Phase 2 which won't experience that 4-6 hours of downtime.

Posted: Wed Jun 04, 2008 10:16 am
by Jorg
Hi Tierra,

Thanks a lot for the explanation. I kind of thought there was some problem again, but you always hope it is over soon. I don't think the forum itself had any damage by the downtime as it seems to run fine again, and people are happily posting away like nothing happened.

Jason did not respond I think he is really busy helping his paid customers to get their sites in the air, or he's not working at the moment. In any case, I'm glad it seenms to be resolved. Let's hope the generators hold until everything is a-ok there again.

- Jorgen

Posted: Wed Jun 04, 2008 6:18 pm
by jsheets
Good morning, yes the server is hosted at The Planet and unfortunately was affected by their outage.

They were able to bring the servers back up on generator power but unfortunately the generator failed, leading to the additional outage.

At this time they are still running on generator power so at some point down the road there will be another, hopefully brief, period of unavailability when they switch back to main power.

Posted: Wed Jun 04, 2008 8:51 pm
by Jorg
Hi Jason,

Thanks for the explanation and ofcourse we're ever thankful for your hosting efforts for the wxForum. If we have outage now and then (even when a freak accident like this occurs) we'll patiently wait until it is over ;-)

Thanks,
- Jorgen

Posted: Fri Jun 06, 2008 6:38 pm
by jsheets
Hello all, The Planet is continuing to have problems and honestly is killing us. We are in the process of moving the remaining server away from them.

Over the next few days I will be coordinating with Jorg and other site admin to migrate wxforum to another server.

The other server should be somewhat faster as it has more modern hardware including dual core processor and more memory.

Posted: Thu Jul 17, 2008 1:23 pm
by doublemax
i'm experiencing downtimes pretty often lately. Are you currently in the progress of moving it to another server? If not, are you still planing to do so?

Posted: Thu Jul 17, 2008 1:27 pm
by upCASE
doublemax wrote:i'm experiencing downtimes pretty often lately. Are you currently in the progress of moving it to another server? If not, are you still planing to do so?
Yep, same for me.
No, we're not moving, unless Jorg decided to do so :)

Jorg: I didn't have time to mail you, but did you contact Jason about the downtimes?

Posted: Thu Jul 17, 2008 6:54 pm
by jsheets
Hello all, I will be moving wxForum to a new server this evening. I have decreased DNS TTL, and will also be setting up an additional alias for those that wind up with cached DNS.

Posted: Thu Jul 17, 2008 7:35 pm
by doublemax
that's good news - much appreciated =D>

Posted: Thu Jul 17, 2008 9:38 pm
by Jorg
Hi Jason,

Thank you very much. I am sorry I saw your message too late I have been sick and a bit tired from sleeping bad (from geing sick) so I wanted to contact you about steps to take, but it seems you beat me to the punch.

Thanks again for your great efforts to host this forum on your domain!

With regards,
- Jorgen

wxForum Move Complete

Posted: Fri Jul 18, 2008 2:50 am
by jsheets
I've moved the site to the new server and everything appears to be well.

Some users may still have cached DNS entries but they should timeout soon.

The site should may even be somewhat faster as this server is considerably better.