The Recent Downtime

By Dominic Mulligan on October 22, 2011 - 22:48

So, you may have noticed a nice big chunk of downtime earlier. During the 3 hours the site was inaccessible, chaos ensued behind the scenes. The site was practically almost deleted and restored several times, and more than one desk was punched in the process. The story of the downtime is as follows.

I was cleaning up the site, moving things around and generally tidying up things on the back-end when I discovered that some shows could not be edited. I messed with a few settings and it became apparent that the site would need to be taken down to be fixed. The next three hours involved a lot of hair-pulling and browsing of the drupal forums. In the end, it turned out that the culprit was a custom-rolled RSS solution that we had hacked together. I'm not sure why, or even how, it became a problem, and I don't why we didn't notice it before now.

Either way, the site has been restored, and I will need to investigate a new RSS solution.