This post originated from an RSS feed registered with Agile Buzz
by Jared Richardson.
Original Post: A Great Test
Feed Title: Jared's Weblog
Feed URL: http://www.jaredrichardson.net/blog/index.rss
Feed Description: Jared's weblog.
The web site was created after the launch of the book "Ship It!" and discusses issues from Continuous Integration to web hosting providers.
There are several characteristics a great test should have. I've discussed the topic recently and decided to create a blog entry about it. It will also be a part of a book I hope to start early next year.
By the way, bear in mind that I'm not talking about unit tests. While they do share some characteristics, they don't require external resources like external data.
Create
First, the test (or it's caller) configures it's environment. If the test needs data in a database, wipe the old table, recreate the schema, and re-add the data. This can be done once per test run (as opposed to once per test) but leaving old data laying around is just asking for data integrity issues to cast doubt on the resutlts your tests returns.
Anything you can do to make your tests rock solid and relable, you should do. Recreating data for each test run is an important step. Use tools like dbUnit to easily reproduce your data.
Run
A great test runs.
That is to say, nobody has to run it. Once a test suite is set in motion, nobody needs to push a button, start a server, click a dialog or run a script.
Nothing is more frustrating that seeing someone create an "automated" test that doesn't run without human intervention. Sometimes people get so close, but forget to stop short before the edge of the cliff.
Decide
Next, a great test passes or fails. A mediocre test has you look at log files to determine the pass or fail status.
Build enough smarts into your test that it can either pass or fail without your help. It already has enough smarts to test the code. Add just a bit more and let it know what the result is. This frees you from the boring chore of reading log files every morning.
Fail
A great test also fails from time to time. If it never fails, then it never catches a problem. You still get a lot of benefit from knowing your code is solid, but push the envelope with some of your tests.
There are few ways to move in this direction.
You can target active code development. Put a few tests in place before the refactoring starts. These tests will ensure that your work doesn't change the code's function. This is great way to start your performance tuning, bug fixing, or feature adding work.
Wrap tests around bugs as you find them and then test the code close by.
Target code that's had historical problems. The odds are good that the routines that caused problems last year have undiscovered issues. Get test coverage in place to help uncover those unknown issues as well as keep the state of the code stable when addditional bug fixing occurs.
Clean
Finally, a great test cleans up after itself. It doesn't leave test droppings throughout the system that could corrupt the next test run or have unintended consequences.
Report
A great test leaves behind enough of a message to determine where the failure occurred and what caused the problem. A great test returns more than a stack trace. It also returns a short message that tells you exactly what it looked for and didn't find.
Sure, someone could go code diving to find the failed test and attempt to decipher the intent of the author. But a great test doesn't require that. It tells you what happened.
More?
This is a start... what more? What do you think about when you write a great test? What annoys you in a bad test? What test patterns trap the mediocre tests before they reach greatness?
Previous Topic
