Summary
The latest version of the Java unit testing framework includes new a syntax for expressing assertions, and allows developers to implement assumptions and theories in tests.
Advertisement
The JUnit project released the 4.4 version of the popular Java unit testing framework. In the project's release notes, project leaders David Saff and Kent Beck note that:
JUnit is designed to efficiently capture developers' intentions about their code, and quickly check their code matches those intentions.
Over the last year, we've been talking about what things developers
would like to say about their code that have been difficult in the
past, and how we can make them easier.
Among the features introduced in this version is a new assertion syntax originally developed by Joe Walnes. The new syntax has the form:
assertThat([value], [matcher statement]);
This makes statements such as the following possible:
The advantages of this feature, according to the JUnit 4.4 release notes are a more readable syntax, better error messages, and the ability to combine matchers for a particular condition.
The new assertion syntax also enables two additional JUnit 4.4 features: assumptions and theories. Of the former, Saff and Beck write that:
Ideally, the developer writing a test has control of all of the forces that might cause a test to fail.
If this isn't immediately possible, making dependencies explicit can often improve a design.
For example, if a test fails when run in a different locale than the developer intended,
it can be fixed by explicitly passing a locale to the domain code.
However, sometimes this is not desirable or possible.
It's good to be able to run a test against the code as it is currently written,
implicit assumptions and all, or to write a test that exposes a known bug.
For these situations, JUnit now includes the ability to express "assumptions:"
import static org.junit.Assume.*
@Test public void filenameIncludesUsername() {
assumeThat(File.separatorChar, is('/'));
assertThat(new User("optimus").configFileName(), is("configfiles/optimus.cfg"));
}
@Test public void correctBehaviorWhenFilenameIsNull() {
assumeTrue(bugFixed("13356")); // bugFixed is not included in JUnit
assertThat(parse(null), is(new NullDocument()));
}
Beck and Saff also note that in the JUnit 4.4 release, a failed assumption will still let a test pass, but in future versions this behavior may change.
Noting that,
A test captures the intended behavior in one particular scenario. A theory captures some aspect of the intended behavior in possibly infinite numbers of potential scenarios.
JUnit 4.4 introduces "theories," a feature borrowed from the Popper project.
@RunWith(Theories.class)
public class UserTest {
@DataPoint public static String GOOD_USERNAME = "optimus";
@DataPoint public static String USERNAME_WITH_SLASH = "optimus/prime";
@Theory public void filenameIncludesUsername(String username) {
assumeThat(username, not(containsString("/")));
assertThat(new User(username).configFileName(), containsString(username));
}
}
Defining general statements in this way can jog the developer's memory about other potential data points and tests, also allows automated
tools to search for new, unexpected data
points that expose bugs.
Sheesh - all four of the examples are just as easy, and far less confusing, to implement in the old-fashioned plain old Java syntax. For example - is the new JUnit either/or syntax for responseString "or" or "xor"? In Java I know.
All this "improvement" is just an improvement for JUnit consultants and more confusion for programmers. Why should I have to learn a whole new syntax for Boolean operations when Java has a perfectly good one? There's a reason mathemeticians developed symbols, not English to represent Boolean ops.
Stick to Plain Old Java Syntax (POJS). It's called JUnit, not CoolNewSyntaxUnit.
> > assertThat(x, is(3)); > assertThat(responseString, > either(containsString("color")).or(containsString("colour") > > assertThat(myList, hasItem("3")); > > > And how the heck is this is an improvement over > > > assertEquals(3, x); > assertTrue(responseString.contains("color") || > responseString.contains("colour")); > assertTrue(myList.contains("3")); >
The most immediate benefit is that the error messages from the second group are immediately useful. See more information at:
It would be great if we could extract such useful error messages while still using Java's native operators (this would be possible, for example, in LISP). With the current design of the Java language, however, we've decided to leave the trade-off up to individual test writers.
I hardly ever look at the error messages, just go to the code where it failed. Unless I wrote and still remember that test, I'll not have any idea where or why "colour" was expected and how "something else" was generated.
Or, most likely, the test used to work, and recently started failing. I'd want to look at / debug through the code thats getting called to see what changed recently.
> I hardly ever look at the error messages, just go to the > code where it failed. Unless I wrote and still remember > that test, I'll not have any idea where or why "colour" > was expected and how "something else" was generated.
Consider
assertThat(responseString, anyOf(containsString("color"), containsString("colour")));
// ==> failure message:
// java.lang.AssertionError:
// Expected: (a string containing "color" or a string containing "colour")
// got: "Please choose a font"
In my own experience, I've really appreciated seeing right in the error message what the value of responseString is, without having to open the debugger or execute the target code in my head.
Lets assume that when you see ""Please choose a font" instead of "Colour" you know exacty what is wrong. Which is possible (though, IMO, not all that likely). What are you going to do?
a) You remember that Fred wrote the text properties code, assign him the bug. b) Fred jumps into the code, fixes the bug, verifies it by rerunning the unit test, commits.
Let's say you have No error message
a) You notice that the error is in TestTextPropertiesDialog, Fred wrote TextPropertiesDialog, so you assign him the bug. b) just like in case 1.
I still don't see any benefit.
Can you elaborate on the later parts of the article (assume, theory, etc.) Those sounded more interesting and potentially useful to me.
> Lets assume that when you see ""Please choose a font" > instead of "Colour" you know exacty what is wrong. Which > is possible (though, IMO, not all that likely). What are > you going to do? > > a) You remember that Fred wrote the text properties code, > assign him the bug. > b) Fred jumps into the code, fixes the bug, verifies it by > rerunning the unit test, commits. > > Let's say you have No error message > > a) You notice that the error is in > TestTextPropertiesDialog, Fred wrote TextPropertiesDialog, > so you assign him the bug. > b) just like in case 1. > > I still don't see any benefit.
Let's assume you're Fred. Does that change the scenario?
Using the new syntax has improved my development experience. If a text properties dialog fails, one error message can tell me if:
1) I'm returning a null String. 2) I've somehow misspelled "coulor". 3) I'm accidentally talking about fonts instead.
Getting this for free is worth learning and using a different syntax for me. I can easily imagine that it would not be worth it for some people, for which all of the old assert methods will still be supported forever.
> Can you elaborate on the later parts of the article > (assume, theory, etc.) Those sounded more interesting and > potentially useful to me.
- Even having read through the new hoola release notes, the thing doesn't give me anything new that couldn't have been expressed with what's already been there. - It's also very doubtful that the new API is more meaningful. It might be to some, to others (including me), it's not. - And MOST OF THE TIME, the person sifting/changing some code of released software is not the one who's originally written it. That's why you should provide your own explicit failure text rather than relying on generic default output provided by a library.
* There problem to me is not that JUnit has a new API as it still supports the old one. My problem is that it's got a new dependency. And with that, my project all of a sudden has a new dependency to some Hamcrest party, too!
> Stick to Plain Old Java Syntax (POJS). It's called > JUnit, not CoolNewSyntaxUnit.
I think Morgan's got a point here. If users are going to break with plain Java syntax, why not use a fully-featured language that supports scripting (e.g. Groovy). Personally, I think that makes more sense anyway. It seems to me the main advantage of using Java for test scripts is that nobody is forced to leave their Java cocoon.
> JUnit now includes the ability to express "assumptions" : > With this release, a failed assumption will lead to the > test being marked as passing, regardless of what the > code below the assumption may assert. In the future, > this may change, and a failed assumption may lead to the test being ignored
Is this saying that a new feature is being introduced that currently does nothing and whose future behavior may become version dependent? It sounds like an odd strategy.
In addition, I'm wary of a feature that potentially may lead to tests being ignored. Unless a test is commented out, it would seem to me more logical that tests should either pass or fail.
While I agree this would read better and be more inline with fluent interfaces, I think it's trickier to implement.
Keep in mind, that is(), not(), etc, are all static imports. So, you could write your own and statically import them, and go off without much hassle.
To use assertThat(something).is(somethingelse), is() would have to be a method on an object that assertThat() returns. So to add your own methods, you'd have to make a subclass of whatever assertThat() might return, and you'd have to have a way to make sure that assertThat() is returning your new subclass. I can't even think about how you might go about achieving that
This seems like it would take away a lot of the flexibility of using static imports provides you.
> Lets assume that when you see ""Please choose a font" > instead of "Colour" you know exacty what is wrong. Which > is possible (though, IMO, not all that likely). What are > you going to do? > [...] > > I still don't see any benefit.
It's like including a detail message or cause when you throw an Exception. It often makes maintenance easier, especially for an intermittent problem or nondeterministic test.
This is not just an issue with the new API. I hate going to a test failure and finding something like:
assertTrue( foo == 42 );
instead of
assertEquals( 42, foo );
Why did the test fail? What was foo? Why should I need to go into the debugger just to find out what foo was? I always change these to assertEquals().
Someone else suggested using the message for this:
assertTrue( "foo should be 42 but was " + foo, foo == 42 );
but that is redundant code which increases the maintenance cost with no advantage over assertEquals().
The assertEquals() API is more appropriate and the better choice. I imagine that this new API will also be the better choice, for the same reason.
Flat View: This topic has 19 replies
on 2 pages
[
12
|
»
]