aaron c beals

this is a story about testing, or the lack thereof.

for my ‘day job‘ (oddly named since i often find myself working on it at night), i manage a website that is used by thousands of health care professionals around the world. in order to keep things running smoothly on the site, we have a series of tests that we run every time we make a change to the code powering the site. these tests let us know that the code we have just written (often for a new feature) hasn’t broken the expected behavior of existing functionality.

these tests are Very Important. i cannot stress this enough. if you are a software engineer, a software QA engineer, and especially if you are a support engineer, i am preaching to the choir.

i’m not saying that our test suite is perfect. but we have one, and we add to it regularly. we do this because the penalty for failure to test can impact thousands of users and in the worse case, drive them away from using our site.

i also run my own site. for many years, i wrote the code behind the site myself, but over time, i started to realize that i was essentially building from scratch what the folks over at WordPress had already implemented quite well. since they test their own code, i had no incentive to create a test suite for my site or my wife’s blog.

about 6 months ago, i upgraded WordPress from the absurdly old 2.0.x to a more modern 2.7.x — an upgrade which went seamlessly and also allowed me to use their new one-click upgrade feature (i.e. no more unpacking tarballs and editing config files). however, the feature was DOA.

after some digging, i found that i needed to move to MySQL 5 in order to take advantage of the quick-upgrade feature. the data migration wasn’t painless due to the vagaries of our hosting provider, but after some massaging, i got things ported over. next, i found that an upgrade from PHP 4 to PHP 5 was required — luckily, i just had to add a line to the .htaccess file in the root directory. with a few other minor tweaks, we were good to go.

earlier this month, my wife tried to log in to the administration panel of a custom CMS she purchased a few years back. the login failed, as did the contact form submissions she had been receiving for a few months. i looked into the issue and found portions of the CMS code were not PHP 5 compatible (despite 5 being theoretically backwards-compatible with 4). turns out that adding the .htaccess file to the root directory, as my hosting provider had recommended, caused PHP 5 to be used in all subdirectories.

why didn’t i catch this earlier? well, i was making changes i thought only impacted our WordPress installations. i had not touched other databases or code installations. since my wife’s wedding website appeared to work, i did not think to test features like the CMS admin panel or contact form.

why was it a big deal? if the issues had just impacted our blogs, it would not have been. however, the contact form on my wife’s wedding photography website is used to field questions by prospective clients. empty emails mean missed opportunities. the good news is that she’s putting her business on hold while we adjust to becoming parents, but this sort of mistake is still relatively costly.

would testing have helped? certainly testing of some sort (manual) would have helped. automated testing would have been harder to set up, since we had a bunch of different interfaces, but something like Selenium could have done the trick.

why don’t we have automated tests for our personal websites? discuss.