This post originated from an RSS feed registered with Java Buzz by Simon Brown. Original Post: Who tests the tests? Feed Title: Simon Brown's weblog Feed URL: http://www.simongbrown.com/blog/feed.xml?flavor=rss20&category=java Feed Description: My thoughts on Java, software development and technology. Latest Java Buzz Posts Latest Java Buzz Posts by Simon Brown Latest Posts From Simon Brown's weblog

Being a blogging application, Pebble is quite date/time sensitive, particularly when it comes to generating things such as permalinks. Much of the code has been running without problem for a long time now (did you know that Pebble was started 3 years ago?) and I have a fairly decent suite of tests. Recently, somebody reported a problem about the unit tests failing. This was odd because "they work for me", but odder still was that the tests were only failing in the afternoon, after 4pm.

Since I live in the UK, all of the development work and testing happens in the GMT/BST timezones. And my host, well that's located in Canada so I was fairly confused when somebody had reported a bug that suggested there were problems related to the time zone in which the code runs.

After a bit of digging around, I found the problem. I was trying to be a little bit cleverer in my unit tests than perhaps I should have been. All of the failing tests had the same root cause - a java.text.SimpleDateFormat instance was being used as a quick and easy way to generate the expected result of a test. Inside of Pebble, there are lots of places that date formatters are used but there's one crucial difference - they've been made aware of an explicit time zone whereas the formatters in the tests hadn't.

This has taught me an important lesson. Either (a) be very careful when generating dynamic values for expected test results or (b) just make the tests more explicit through the use of known data values. Test the tests, or keep them simple.