Summary
Martin Fowler recently published a new version of his classic article on testing with mock objects. He distinguishes between tests that rely on behavior verification, and tests that verify an object's state, and highlights the difference between classic "TDDers" and "mockists." To which camp do you belong?
Advertisement
Martin Fowler recently penned a new version of his article on two styles of unit testing: one based on the classical test-driven development model, and the other based on state verification using mock objects: Mocks Aren't Stubs.
In the article, he notes that when writing tests,
You're focusing on one element of the software at a time—hence the common term unit testing. The problem is that to make a single unit work, you often need other units.
Fowler then dives into the various ways developers can provide the other objects unit tests require, highlighting terminology introduced in an upcoming book by Gerald Meszaros:
Meszaros uses the term Test Double as the generic term for any kind of pretend object used in place of a real object for testing purposes. The name comes from the notion of a Stunt Double in movies. (One of his aims was to avoid using any name that was already widely used.) Meszaros then defined four particular kinds of double:
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in-memory database is a good example).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it "sent," or maybe only how many messages it "sent."
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
Of these kinds of doubles, only mocks insist upon behavior verification. The other doubles can, and usually do, use state verification.
The rest of Fowler's article goes into the distinction between tests that rely in behavior verification, and test that verify an object's state. Because the two approaches lead to different styles of tests, Fowler notes that,
The big issue here is when to use a mock or other double...
The classical TDD style is to use real objects if possible and a double if it's awkward to use the real thing. So a classical TDDer would use a real warehouse and a double for the mail service. The kind of double doesn't really matter that much.
A mockist TDD practitioner, however, will always use a mock for any object with interesting behavior. In this case for both the warehouse and the mail service...
The first thing to consider is the context. Are we thinking about an easy collaboration ... or an awkward one...? If it's an easy collaboration then the choice is simple. If I'm a classic TDDer I don't use a mock, stub or any kind of double. I use a real object and state verification. If I'm a mockist TDDer I use a mock and behavior verification. No decisions at all.
If it's an awkward collaboration, then there's no decision if I'm a mockist—I just use mocks and behavior verification. If I'm a classicist then I do have a choice, but it's not a big deal which one to use... The real issue is between classic and mockist TDD.
What style of tester are you: A classic TDDer or a mockist?
Honestly, Martin Fowler has made his career confusing the hell out of developers. What is a Mock and why are we pretending like anyone has ever heard of this term before. Maybe now we can all scramble and google it, then we can pretend like we know what he's talking about and be the coolest pattern dude on the block. None of this stuff is for developers that accomplish real work.
You're entitled to your opinion, Adam, but I used TDD and unit-testing on my last project, and the bug count was way down compared with previous projects. That was using C; now I'm looking to do the same in C++ (and maybe C# too), so I'm looking carefully at mocks. Stubs are fine for C, but not so helpful for C++. Maybe you don't work with OO languages?
I'll tell ya, it's an absolute breeze in C#. C++ is a little tougher to get started with as there's a bit more typing per test. But really, once you're on your way it's wonderful.
> Honestly, Martin Fowler has made his career confusing the > hell out of developers.
I've found some things he's written to be a little less than completely clear too. It's generally because he uses a lot of esoteric terminology. I think he's a really smart guy, just not the best at explaining things.
> What is a Mock and why are we > pretending like anyone has ever heard of this term before.
I've heard of it and I use the term. 'Mock' the general term is loosely equivalent to 'fake' or 'phony'. Mock Objects are essentially stand-in Objects that replace the 'real' or production Objects. They only make sense if you have a decoupled design.
I am using Mock Objects to test a message handler I wrote for a messaging system. I cannot use JMS and the Java APIs for this queueing system are pretty crappy. They are very C-esque. In any event, I created a Handler interface for the actual business logic that receives the message and does something with it.
Where the mocks come in is for testing the code that reads from the queue. I create a bunch of mock-handlers than do different things when they receive the message. One just takes the message and when it's committed, it compares what it has with every other instance of that class and logs an error if it didn't match. Another one throws an exceptions on the receipt of the message. Another one throws an exception when commit is called. Yet another throws an exception when a rollback is executed. These handlers make it easy for me to test how the code handles these situations and make sure it fits my design.
I've been doing pretty straight TDD due to the opacity of the libraries on the projects with which I've been working.
I'm definitely going to be using some form of shallow mock scaffolding as the complexity of what I'm testing increases. But the full-blown "contract" style "expected/received" stuff I'm seeing just goes too far in that direction for me.
But then, I'm pretty sure that's exactly what I thought about TDD to begin with :)
As I'm fairly new at this, I'm still working on finding the line between "testable" and making everything public.
It strikes me that trying to get significant ratios of code coverage under unit tests without mocked up interfaces would just require far too much destruction of the OO structure of the code in order to provide access to tests.
Or is that one of those "ah, you haven't been doing this long enough to see what really happens. Use the force." moments?
Mock objects are very interesting. I have found them to be very useful when first laying out the design of a feature, but once that is done they become kind of a boat anchor, since even the most trivial tweaks to the code often need to be done in two places: the code itself and the expectations.
I think that they are much closer to a design verification tool, than a unit test one -- kind of like the executable embodiment of a sequence diagram. In fact, it would be interesting to see someone build a tool that could forward-reverse engineer sequence diagrams to mock object test cases rather than actual code.
> It strikes me that trying to get significant ratios of > code coverage under unit tests without mocked up > interfaces would just require far too much destruction of > the OO structure of the code in order to provide access to > tests. > > Or is that one of those "ah, you haven't been doing this > long enough to see what really happens. Use the force." > moments?
In my estimation, unit tests are good for classes that meant to be passed around and poked at. You have some setters maybe and you call a method and it returns a value. Mock Objects are good for testing classes that coordinate the activities of other Objects. Where stubs come is is when a class that you are testing requires some external resource to work. The most common would be a database. You can set up a dev database to deal with this but that makes the test hard to repeat in a build script, for example. If you build a stub database class that returns data from a local file or something like that, you can make your test stand-alone.
I think that's the distinction that Martin Fowler is trying to make here. You don't create stubs that return garbage data. They aren't used for testing directly like mock objects.
> Mock objects are very interesting. I have found them to be > very useful when first laying out the design of a feature, > but once that is done they become kind of a boat anchor, > since even the most trivial tweaks to the code often need > to be done in two places: the code itself and the > expectations.
Isn't it the same for unit tests? I have found unit tests to be much worse with regard to the problem you are describing.
> I think that they are much closer to a design verification > tool, than a unit test one -- kind of like the executable > embodiment of a sequence diagram.
That sounds like a good description to me.
> In fact, it would be > interesting to see someone build a tool that could > forward-reverse engineer sequence diagrams to mock object > test cases rather than actual code.
Do you use sequence diagrams much? I find them so time consuming. I only use them at a very high-level, like for describing a web-service.
> Honestly, Martin Fowler has made his career confusing the > hell out of developers. What is a Mock and why are we > pretending like anyone has ever heard of this term before. > Maybe now we can all scramble and google it, then we can > pretend like we know what he's talking about and be the > coolest pattern dude on the block. None of this stuff is > for developers that accomplish real work.
I agree wholeheartedly with you. The more I read his blog, the more I feel that he is trying to generate more revenue for his company by appearing to be very smart, not that he isn't somewhat smarter than the average developer.
His best works were Analysis Patterns and Refactoring. But his latter books, especially the "Patterns of Enterprise Application Architectures" seems to have been written at breakneck speed and rushed to the printers, to ensure that soemthing gets published before "patterns" goes out of fashion.
Meszaros uses the term Test Double as the generic term for any kind of pretend object used in place of a real object for testing purposes. The name comes from the notion of a Stunt Double in movies. (One of his aims was to avoid using any name that was already widely used.) Meszaros then defined four particular kinds of double:
This is just renaming the general concept of stub. A stub is any object or module which implements a public interface without neccessarily providing all of the final application logic for the purposes of prototyping and testing during development. (I am open to corrections here).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it "sent," or maybe only how many messages it "sent".
This is not a definition of stubs, it is merely an observation of a common idiomatic usage of stub.
I think it would have been wiser to simply label the idiom, rather than to attempt to redefine the general case.
Let me know if you think I'm out in left field here.
> Let me know if you think I'm out in left field here.
I think it's an attempt to refine some terminology. Obviously, you don't have to accept it. Actually, after a closer reading of the article, his definition of a 'mock' is not what I described above. I guess what I was calling a mock object is what he calls a 'stub'.
Martin Fowler IMHO is a really clever bloke, I don't agree with everything he says but he at least backs his ideas with logical thoughts and reasons rather than a religious or emotional statement about how things feel and what he likes/dislikes. I'll go so far as to say there IS a lot of hype surrounding agile/xp/tdd practices with many people insisting they've discovered the only way to develop software but Martin certainly isn't one of these. I use some of "this stuff" and I happen to accomplish real work so next time you hit the reply link you'd do well to think about what you're typing and backing up your notions with some solid reasoning/evidence instead of making a fool of yourself like you have just now.
Of these kinds of doubles, only mocks insist upon behavior verification. The other doubles can, and usually do, use state verification.
I do think this is the only relevant distinction Fowler makes in his article and I'm also confused about the other distinctions he quotes. For "behaviour verification" in the testphase you don't even need to mock anything but add some monitoring facility for recording method calls just like a profiler records timing information. Everything can be programmed using a tracing interface which is available for any reasonable programming language implementation. Not sure I want this and whether checking slices of a call graph is a relevant invariant? Never heard about anyone who rewrites all his UTs when renaming a method ;)
Maybe my C++ and Python experience has influenced me because in those languages you can write tests that get inside the internals of classes without a public interface, checking state (using friends in c++). Thus the objects become their own stubs which simplifies writing tests.
A unit test then becomes a way of checking the pre and post-conditions without building them into the method being called.
MockRot is a real concern but I think also presents an opportunity as a new metric.
The rate of change of Mocks might just indicate a change in understanding of behaviour ie: better testing.
More interestingly, it could expose that a design is unstable with regards to object behaviour. This latter point may be hard to isolate with existing metrics aimed only at the original classes being changed.
As for Adam Kramer's comments, Fowler makes it clear in the beginning of the essay that he is using Gerard Meszaros's terms. I wish people would read before reacting.
Previous Topic
