LiveJournal is down due to power failure
Temporarily Unavailable
LiveJournal: LiveJournal is currently down due to a massive power failure at our data center. We'll provide updates at /powerloss/ as they're available.
- Our data center (Internap, the same one we've been at for many years) lost all its power, including redundant backup power, for some unknown reason.
- Update #1, 7:35 pm PST: we have power again, and we're working to assess the state of the databases.
- Update #2, 10:11 pm: So far so good.
- Update #3: 2:42 am: We're starting to get tired, but all the hard stuff is done at least.
- Update #4: 9:12 am: We're back at it.
- Update #5: 1:58 pm: approaching 24 hours of downtime... *sigh*
Randy: Yikes! Can't wait to learn from the post-mortem. Reliability of scalable systems.