Weblogs Forum - Are Tests First Class Clients?

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Weblogs Forum
Are Tests First Class Clients?

57 replies on 4 pages. Most recent reply: Aug 2, 2005 9:23 AM by Jim Cakalic

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 57 replies on 4 pages [ « | 1 2 3 4 ]

Vincent O'Sullivan

Posts: 724
Nickname: vincent
Registered: Nov, 2002

Re: Are Tests First Class Clients?

Posted: Jan 28, 2005 12:24 AM

At the end of the day, there are no compulsory rules about how a class should be written (or tested), just lots of sometimes contradictory advice.

In this case it's down to you as the developer to balance the problem of simple API vs simple testability.

My take on it is this. If you do the following...

    public API methods:
        declare as public (no other methods should be public).
 
    test methods: 
        (Fewer is better but that doesn't mean none is best)
        declare as 'package protected',
        Javadoc as 'for testing purposes only',
        name as being for testing purposes,
        return test results (i.e. pass/throw error) not data.

...then you're not exposing any implementation or data and you're having a minimal effect on the public API. You'd be unlikely to introduce any problems worth losing sleep over.

Vince.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Are Tests First Class Clients?

Posted: Jan 28, 2005 1:39 PM

Damn, Vince. That's a great thought.

Name the extra methods testXYZ(), and return/throw a pass/fail signal, rather than data. Nice solution!!

Todd Blanchard

Posts: 316
Nickname: tblanchard
Registered: May, 2003

I'd say you've made an excellent argument

Posted: Jan 29, 2005 11:04 PM

for method categories like we have in ObjectiveC and Smalltalk.

Method categories are methods that are actually associated with a separate package and can be loaded or unloaded on demand. You wouldn't ship your tests in the product so you'd just unload the test method category and there's no pollution in your shipped api.

Of course, you static language victims are out of luck I guess.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Method Categories

Posted: Jan 31, 2005 7:33 AM

> I'd say you've made an excellent argument
> for method categories like we have in ObjectiveC and
> Smalltalk.
>
Another great thought. SO right.
I wasn't aware that there was any such thing!

What's really great about that idea is that it completely solves the problem of where to put test methods. They can be part of the class once again, instead of in a separate class that's named to show the correspondence, in another directory.

unmesh

Posts: 15
Nickname: unmesh
Registered: May, 2003

Re: Method Categories

Posted: Jan 31, 2005 9:43 PM

I think the problem has broader scope than discussed here.
Anytime we want to make a system modular and define relationships between modules, this type of problem is going to come. Tests are just one type of modules which use the main functional module. Method categories solve this problem partially. Aspect Oriented Programming is another approach. We can define advices in Aspectj, and add methods specifically used by tests.
I guess Jim Coplien's research in MultiParadigm Design discusses exactly same issues.
I read some interesting articles here http://www.cs.ubc.ca/~jan/AODPs/. discussing how using Patterns pollutes the the core API and how it can be separated using AspectJ.

Chris Krahe

Posts: 1
Nickname: ckrahe
Registered: Jun, 2003

Re: Are Tests First Class Clients?

Posted: Feb 1, 2005 4:19 AM

Don't let the changes dominate the API and clearly document them as intended for testing.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Method Categories

Posted: Feb 1, 2005 10:26 AM

> I read some interesting articles here
> http://www.cs.ubc.ca/~jan/AODPs/. discussing how using
> Patterns pollutes the the core API and how it can be
> separated using AspectJ.
>
Brilliant. This needs a thread of its own. It never occurred to me that aspects could be used to organize the multi-class features that implement a specific pattern.

Looking at an API, the major question is generally, "Which of these are the methods I need? Is there an Aspect-enabled version of JavaDoc, perchance? If there were, it would separate the methods in a class into the patterns they implement, and provide separate pages for that patterns with links to the participating classes.

Cool stuff. (Apologies to Bill for hijacking this thread.)

Note:
We need a good way to seamlessly launch a new thread from a discussion like this, so that responses like
this one aren't part of the thread we're in, but can be found by traversing a link from the orginal comment. Perhaps we need two buttons? One would say "In-Thread Reply". Another would say "Out-of-Thread Reply", or perhaps "Digression". Dunno. It's a thought.

Tim Orme

Posts: 3
Nickname: backspace
Registered: Jan, 2004

Re: Are Tests First Class Clients?

Posted: Feb 2, 2005 6:52 AM

> To what extent do you feel automated tests should be
> allowed to change the API being tested?

I don't like changing my API just for tests, either. I like the public API to be as spare as possible. Adding "test-only" methods to the public API violates this and potentially causes confusion to the API consumer/client.

In a similar situation to the case described I took the following approach. The class in question was "fronted" by an interface. I added the test-only methods to the class and not the interface. Accessibility for the interface and imlementing class was such that only the interface, and not the implementing class, was available to clients. (Clients could only access the functionality of the class via interface references returned by a factory). The tests I wrote also used interface references, but when I needed to call the test only methods, I cast my reference to the class reference so I could make those calls. A bit of a hack, perhaps, but it keeps my API clean. (The test class, as is typical with JUnit, was in the same package as the interface and its implementing class, allowing it to access the package-visible implementing class.

The whole structure I described above was not coincedental. Where I work the standard is that any business logic class is fronted by an interface, the implementing class has only package level access, and references can only be gained via a factory. Client code is implemented in different packages than where the business logic interfaces and implementation classes reside. At first I thought this was overkill. Most of our interfaces had, and ever would have, only one implementation behind them. But it does provide that nice clean separation between implementation and interface. In some cases you may never actually need it; but when it turns out you do and you already have it in place, it's really nice.

Jan Ploski

Posts: 8
Nickname: jploski
Registered: Aug, 2003

Re: Are Tests First Class Clients?

Posted: Feb 2, 2005 11:12 AM

> Where I work the standard is that any business logic class is
> fronted by an interface, the implementing class has only
> package level access, and references can only be gained via
> a factory.

Amen.

I came up with the same set of rules while studying the impact of dependencies on code modifiability and comprehension. They seem to answer the original question best and as a bonus they reduce the unnecessary coupling between implementations of different classes. Indeed, whole "Inversion-of-Control" frameworks based on that idea are now available (for example, picocontainer.org). Until recently, it was also a prerequisite for using mock objects effectively (as of today, libraries also support mocking a concrete class using dynamic proxies).

Use interfaces. Make each of your classes implement as many of them as necessary. Document contracts in interfaces, not in classes.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Are Tests First Class Clients?

Posted: Feb 2, 2005 11:42 AM

> > Where I work the standard is that any business logic
> > class is fronted by an interface, the implementing
> > class has only package level access, and references
> > can only be gained via a factory.
>
> Amen.
> ...
> Use interfaces. Make each of your classes implement as
> many of them as necessary. Document contracts in
> interfaces, not in classes.
>
I'm glad I'm on this thread. I'm learning a lot.

In a sense, interfaces create a category of user-visible APIs. APIs for testing and internal operations are therefore relegated to the second category of stuff the user doesn't need to see. Very nice!

Craeg Strong

Posts: 1
Nickname: stoneankh
Registered: Feb, 2005

Re: Are Tests First Class Clients?

Posted: Feb 12, 2005 8:45 PM

Sorry I am late to this party.

Over the last year I have been working on a large enterprise-class J2EE project and I have been asked what amounts to this very same question many times.

There is a very simple and elegant answer to this, and it works in every major OO language (Python, Java, C#, etc.).

- put your tests in the same package (but different physical directory) as your production code, that way they have access to package protected methods.

- create a "TestingSubclassXXX" class that is a subclass of your XXX class. This test class provides getters, setters, and other convenience methods as required for verification

- The TestingSubclassXXX class IS NOT the junit class. Rather it is *used* by the junit class. In Java it is particularly elegant: the TestingSubclassXXX class can be an inner class defined within your junit class.

Variants of this work just fine in C#, Python, or C++. The technique is always the same: subclass the class you want to test and run your tests on the subclass. The subclass only adds methods as needed to provide visibility to enable proper verification.

I believe this should solve your problem very simply without having to resort to an inelegant design, or reflection, or some other complex solution. The only compromise you may have to make for the sake of testability is to make some of your attributes/methods protected rather than private.

Eric Armstrong

Posts: 207
Nickname: cooltools
Registered: Apr, 2003

Re: Are Tests First Class Clients?

Posted: Feb 13, 2005 10:53 AM

> The technique is always the same: subclass the class you
> u want to test and run your tests on the subclass. The
> subclass only adds methods as needed to provide visibility
> to enable proper verification.
>
> In Java it is particularly elegant: The TestingXXX subclass
> is an inner class defined within the junit class.
>
Positively brilliant. I'm astonished I didn't think of it.
My only consolation is that I'm in very good company...
:_)

Jim Cakalic

Posts: 2
Nickname: jimc
Registered: Aug, 2005

Re: Are Tests First Class Clients?

Posted: Aug 2, 2005 9:23 AM

> There is a very simple and elegant answer to this, and it
> works in every major OO language (Python, Java, C#,
> etc.).
>
> - put your tests in the same package (but different
> physical directory) as your production code, that way they
> have access to package protected methods.
>
> - create a "TestingSubclassXXX" class that is a subclass
> of your XXX class. This test class provides getters,
> setters, and other convenience methods as required for
> verification
>
> - The TestingSubclassXXX class IS NOT the junit class.
> Rather it is *used* by the junit class. In Java it is
> particularly elegant: the TestingSubclassXXX class can
> be an inner class defined within your junit class.

I don't consider this a solution but an anti-pattern. You're no longer testing the class but the subclass. As a pattern this opens up all kinds of opportunities for overriding the class-under-test's behavior in ways that ultimately subverts the entire testing effort. Aslak Hellesoy has an interesting blog on this topic: http://blogs.codehaus.org/people/rinkrank/archives/000551_oh_no_were_testing_the_mock.html

Really, we should just admit that fishing around in an object to get details for the purpose of test verification is testing of the implementation, not the design intent. The design intent, although difficult to test, could be done through the public API at this point as discussed by Bill Venners in a previous post. It isn't that this is not possible. It's just that it is hard.

Whenever you start testing the implementation you have to be prepared for your tests to break as the implementation is refactored. Good refactoring browsers help to keep the breakage to a minimum. Getting to the values of private fields isn't difficult using reflection. See http://www.onjava.com/lpt/a/4346 for one example. But it does open the door for more breakage down the road. Of course, if your unit tests are fast then you will be running them all the time so you'll see that breakage pretty quickly. And I think it is the penance one should pay for choosing the easy way out and testing the implementation instead of the design intent.

Regards,
Jim Cakalic

Flat View: This topic has 57 replies on 4 pages [ « | 1 2 3 4 ]

Previous Topic

Next Topic