Manual Creation of Characterization Tests

Michael Feathers suggests a simple algorithm for writing characterization tests:

1. Use a piece of code in a test harness.
2. Write an assertion that you know will fail.
3. Run the test and let the failure tell you what the actual behavior is.
4. Change the test so that it expects the behavior that the code actually produces.
5. Repeat.

Let’s assume that I have inherited the maintenance for a sales management system (oh joy!), and that I have to make some changes to the way commissions are calculated. The code below implements steps 1 and 2 in the suggested algorithm (using the JUnit testing framework):

public void testCalculateCommissionDue() {
        assertEquals(-42.42, SalesUtil.calculateCommissionDue(1000.0));
}

The piece of code I am using is the method calculateCommissionDue(), part of the Java class SalesUtil, and I expect my assertion that the commission on $1000.00 is -$42.42 to fail – unless this company has a truly original compensation plan for sales people.

I run the JUnit test (step 3) and get the following failure message:

junit.framework.AssertionFailedError: expected:<-42.42> but was:<200.0>

All right. It looks like for sales of $1000.0, the actual and current behavior says that the commission is $200.0. I don’t know if that’s right or wrong, but it looks like a reasonable value. I have to assume that, if neither the sales people nor the accounting people have not complained so far, the current behavior is what’s expected. Since people are particularly touchy when it comes to money, I’d better make sure that when I start making changes I don’t unintentionally change this behavior. So I modify the test to reflect the actual behavior (step 4):

public void testCalculateCommissionDue() {
        assertEquals(200.0, SalesUtil.calculateCommissionDue(1000.0));
}

I re-run the test and see that it passes. Cool, I have my first characterization test. It’s not much, but it’s a start.

What next? Step 5 says, repeat. Sounds a bit like the instructions on your shampoo bottle. doesn’t it? And if you are a programmer, I bet that you never repeat after you rinse. In this case, however, you do need to repeat because one test is probably not going to cut it. But how many time do you need to repeat? When can you stop testing and start changing the code? It all depends on the circumstances, and there are no easy answers. Michael Feathers provides the following heuristics for writing characterization tests:

1. Write tests for the area where you will make your changes. Write as many test cases as you feel you need to understand the behavior of the code.
2. After doing this, take a look at the specific things you are going to change, and attempt to write tests for those.
3. If you are attempting to extract or move some functionality, write tests that verify the existence and connection of those behaviors on a case-by-case basis. Verify that you are exercising the code that you are going to move and that it is connected properly. Exercise conversions.

Wow. It sounds like a lot of work. Where to start? What am I going to do next with this example?

Well, you’ll have to wait for the next installment in the series, since I promised myself to keep each of part of this series short and sweet. In the meantime, why don’t you try your hand at writing some characterization tests on your own? Download some open source code that you are not familiar with (I suggest something very simple to get started) and see what you can do.