News & Ideas Forum (Closed for new topic posts) - Two Critical Tips for Unit Testing in Java

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

News & Ideas Forum (Closed for new topic posts)
Two Critical Tips for Unit Testing in Java

5 replies on 1 page. Most recent reply: Feb 17, 2003 2:49 AM by RANGAMANA Hermann

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 5 replies on 1 page

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Two Critical Tips for Unit Testing in Java

Posted: Feb 14, 2003 12:22 PM

Two issues that often vex Java developers when they attempt to unit test their code are 1) whether to design their systems in a manner that facilitates testing, and 2) how to test non-deterministic code. A DevX.com article by Bryan Dollery offers tips for handling these dilemmas.

http://www.devx.com/java/Article/9305

Here's an excerpt:

When you write unit tests, sometimes you feel compelled to change your code just to facilitate the test, usually when you need to test a private method or attribute. Doing so is a bad idea. If you ever feel tempted to make a private method public purely for testing purposes, don't do it. Testing is meant to improve the quality of your code, not decrease it.

Having said that, sometimes designing your system in a way that makes testing easier is still necessary. If you need to add a design element to support testing, ensure that it also increases the general quality of the system as a whole. If the design element doesn't, then you're designing your system only to facilitate testing -- which I must stress again is a bad idea.

What do you think of the author's comments?

Steve Conover

Posts: 37
Nickname: sgcjr
Registered: Feb, 2003

Re: Two Critical Tips for Unit Testing in Java

Posted: Feb 14, 2003 5:33 PM

Pretty good article, but I wouldn't categorically rule this out. There are situations where you have whole swaths of functionality that would be left untested if you did nothing - particularly things that are hard to test like persistence (how to do design a test that would REQUIRE you to write code that persists things, unless you go directly to your persistence store and assert whether things are there or not? That's an encapsulation violation arguably almost as bad as changing your code to accommodate tests.

I recently had this issues with tests for persistence. Ultimately I solved it by using a "global" config setting to determine whether to use a mockobject or the real thing, and then changing the global setting back to the "real thing" in tearDown. I'm not really happy with it, but if you're doing TDD, you can't just leave a huge part of your app untested - you'll kill your development velocity later on when you start debugging all the untested stuff.

And anyway, if you have a nice test harness, you can refactor pretty easily. Maybe later down the road, your design will change in ways you couldn't have predicted and you'll be able to repair the ugliness. Or maybe not. In any case I think the correct statement should be "prefer keeping a nice design to customizing it to make it more testable", rather than "don't do it".

Scott Stirling

Posts: 54
Nickname: sstirling
Registered: Jan, 2003

Re: Two Critical Tips for Unit Testing in Java

Posted: Feb 15, 2003 1:51 PM

> When you write unit tests, sometimes you feel compelled
> to change your code just to facilitate the test, usually
> when you need to test a private method or attribute. Doing
> so is a bad idea. If you ever feel tempted to make a
> private method public purely for testing purposes, don't
> do it. Testing is meant to improve the quality of your
> code, not decrease it.

I agree with the point, but would love to see more in depth explanation of why its converse ia a bad idea. I think the piece is from a breezy advice column, and is excellent as such, but the topic deserves much deeper consideration and investigation. The person who follows this bit of advice without knowing why he should is just following directions and missing out on the bigger picture.

> Having said that, sometimes designing your system in a way
> that makes testing easier is still necessary. If you need
> to add a design element to support testing, ensure that it
> also increases the general quality of the system as a
> whole. If the design element doesn't, then you're
> designing your system only to facilitate testing -- which
> I must stress again is a bad idea.
>
> What do you think of the author's comments?

In the JUnit yahoo group recently we've been discussing the topics of testability and what it is that TDD does to ensure it without intentionally being too concerned about it.

I've tried to capture some of the underlying phenomena in my blog and posts to the yahoo group:

http://users.rcn.com/scottstirling/2003/02/13.html#a2
http://users.rcn.com/scottstirling/2003/02/13.html#a3

There are those who point to motherboard, electronics, and other manufacturing and engineering examples where test points, jumpers, and leads may be exposed to aid testability. So why should software be different? But my concern is with coming up with the best design.

There probably is no one best design for many problems -- i.e., two or more completely different implementations might be just as testable, maintainable, minimal, modular, and fast. What we have so far in software design are a lot of patterns, idioms, best practices, general recommendations and tips, i.e., practical knowledge. I think we need a better understanding of the general principles of good object oriented design, i.e., the theory underlying the practice. I think Robert Martin's recent book (Agile Software Development: Principles, Patterns and Practices) and his papers are the closest thing we have right now to a solid set of first principles.

But generally, I think it's good that practice precedes theory (just as nature precedes science). We have plenty of computer science theory, but most software design and development gets done on the job. Hopefully by investigating the principles underlying good practices we can better understand what practices, methodologies and tools work best and why.

Mike Spille

Posts: 25
Nickname: mspille
Registered: Nov, 2002

Re: Two Critical Tips for Unit Testing in Java

Posted: Feb 15, 2003 2:55 PM

I think the overall thrust of the article was a good one - don't allow small niceties that facilitate testing to overwhelm your true functional & non-functional application reuirements. And try to keep your tests as close to the real application use as feasible. This avoids the disastrous consequence of proving your code works in a highly artificial development environment, without saying anything about how it will work when integrated into the true system.

That said, I have a few nitpicks. Primary among them is propogating the idea that "Unit tests are so named because they test a single unit of code. In the case of Java, a unit usually equates to a single class". In my experience this approach doesn't scale well to large scale development of complex systems.

In my own work, units typically work out to be what I call a "sub-system" which accomplishes a well-defined unit of work. Sub-systems may be one class, but more often comprises a "public" interface with a small number of worker classes hidden behind the interface. Public here is in quotes because it may not be truly public in the sense of an API, but instead there's an interface that other's can use, and internal details that can change at will. In this setup my unit tests concentrate on exercising the public interface.

The bang for the buck here is that I'm testing a logical unit of work that has meaning within the application, and this logical unit is a "component" that will be integrated into the larger system. And when I need to refactor, I can change the hidden worker classes at will without impacting the unit test at all - because the unit test only tests the externally observable behavior.

The problem with the "unit test per class" method is that classes rarely match a logical unit of work. Indeed, a typical class is usually an implementation detail that should remain hidden. When you refactor what I call a sub-system in this sort of scenario, you have to refactor your individual class unit tests as well - which means you're stuck in a double-change situation: you're changing your tests and your code at the same time, and therefore in the case of errors can't be confident where the fault lies, in the changed tests or the changed code.

The other big nit I have is the area of non-deterministic code. Let me say first that I love the fact that this subject is broached at all - far too many people seem to ignore this aspect, despite the fact that many developers today are dealing with multi-threaded solutions in server side software. But I think he's sending the wrong message. In one example, it says:

"For instance, you could have a requirement that 90 percent of all Web-based transactions should complete within 1/100th of a second. Your technique here would be simply to run the test code many times (in a loop) and compare the transaction times with one second, keeping track of the number of passes and fails. If at the end of the test less than 90 percent of the transactions fail, then the test fails too."

The above may just be a poor example, but I can't tell from the text. The problem here is that the author is stating a non-functional performance requirement - and then indicates that a single-user, single-threaded test is sufficient to gauge whether you've hit the requirement or not.

In my experience any performance requirement involving server-side code (like a web-based transaction environment) needs to at least attempt scalability over multiple users in addition to simple single-user timings. While the single-user test outlined has value in baselining the "unconstrained" behavior of the system, a conscientious developer should run it at least at 5 & 10 simulated users to see if it still meets requirements e.g. do 10 users each hitting the system simultaneously meet the 10 millisecond requirement 90% of the time?. (as an aside, I'm glad I don't work in an environment where Internet server response time has to be at 10 millis or under!).

As I said, overall the article is a refreshing change from the usual unit test mantra, but some of the details detract from the message.

-Mike

Mike Spille

Posts: 25
Nickname: mspille
Registered: Nov, 2002

Re: Two Critical Tips for Unit Testing in Java

Posted: Feb 15, 2003 3:04 PM

>
> There are those who point to motherboard, electronics, and
> other manufacturing and engineering examples where test
> points, jumpers, and leads may be exposed to aid
> testability. So why should software be different? But my
> concern is with coming up with the best design.
>

I happen to agree that software should be instrumented with "test points", just like hardware is. This is a tough sell in today's environment, though, because test points aren't about unit testing at all - they're most useful when running integration tests, performance tests, or diagnosing problems in production.

Look at configurable logging systems, where you can switch on different packages and debug levels on the fly. Or hooks for monitoring systems that can report the system internals in realtime. Such instrumentation in your code is invaluable to determine what the system is doing in complex enviroments with unique loading situations. But people are so fixated on unit testing these days at the class level that more comprehensive testing often gets overlooked. And IMHO these comprehensive test points have a much larger impact on overall system quality than extremely fine grained unit tests.

-Mike

RANGAMANA Hermann

Posts: 5
Nickname: hrangamana
Registered: Feb, 2003

Re: Two Critical Tips for Unit Testing in Java

Posted: Feb 17, 2003 2:49 AM

Flat View: This topic has 5 replies on 1 page

Previous Topic

Next Topic


	Web Artima.com