This post originated from an RSS feed registered with Agile Buzz
by Laurent Bossavit.
Original Post: Fitnesse, I almost knew you
Feed Title: Incipient(thoughts)
Feed URL: http://bossavit.com/thoughts/index.rdf
Feed Description: You're in a maze of twisty little decisions, all alike. You're in a maze of twisty little decisions, all different.
I've been doodling Customer Tests for a "virtual GoBan", in response to an
old UseNet thread recently resurrected that aimed to compare TDD with a more
formal Design By Contract approach.
As mere doodles, the tests are not executable. I'm writing them using the
tool I favor for all documentation: the ASCII text editor. Reading the tests,
one should form a good idea what the software does; take away a basic
understanding of the board and stones, the rules of the game, and a "theory
of operations" - some model of how the software knows to do what it does.
Here, for instance, is one of the first half-dozen tests, just starting to
lead up to interesting functionality:
# When playing a game, players take turns playing; black always starts
BOARD 2
START
. .
. .
BLACK'S TURN
That style of testing came naturally - I'm sending "commands" to the
application (before the blank line), then somehow getting it to display
its state (game board and an indication of whose turn it is).
Turning the example into an executable test is a matter of parsing the string,
splitting it at the blank line, treating the first part as commands and
verifying that what the program "outputs" at the end is the same as the last
part.
At this point I had a minor brainstorm. I remembered that one of the most
charming Wikis around, Sensei's Library, has a nifty markup format for Go
boards, that can render just about any Go configuration or fragment thereof
to a neat-looking PNG image. This would be particularly appealing for tests
meant to serve as documentation & specification.
Thinking of Wikis, I was primed to think of Fitnesse. Fitnesse is to
acceptance tests what Sensei's Library wiki engine is to Go boards; it's a
domain-specific Wiki. As a Wiki, provides a central repository for documents
that multiple people can edit collaboratively. As an acceptance test engine,
it provides a way to exercise application code against human-readable
examples, and verify them automatically.
So, I invested some time integrating Fitnesse and the Go markup renderer.
And ran into a few enlightening problems.
The first thing I had to do was choose a type of Fixture. In Fit (the
acceptance test engine underlying Fitnesse), Fixtures are somewhat like an
inventory of test grammars to choose from - they are different ways of
representing "if this happens to the program, I want it to respond with that".
There is ColumnFixture, which is useful when the application has several
inputs, and one or more outputs: "if I set the input values X, Y and Z
just so, then it should come back with exactly the speed of light."
There is RowFixtures, used to test the result of queries ("Give me
all X such that Y.") And there is ActionFixture, which corresponds roughly
to the style above: a series of commands, and verifications.
ActionFixture is not, from my present uninformed point of view, the best
feature of Fit; from the little I've seen it seems to do better at
representing regular inputs and outputs, à la ColumnFixture. For script-
like tests (which is what ActionFixtures appear to be), some of the other
acceptance testing tools around might even feel more flexible; Fit's major
constraint (as well as major advantage) is the table-like structure of test
code. It looks great as documentation, but is more cumbersome to write.
The next problem was in converting my test boards to PNG images. What I
wanted was this: when you looked at a test for the virtual GoBan, you'd see
the commands in the left column, and the resulting boards on the right. I had
come across "custom data types" in the Fit documentation: the implied promise
was that, beyond strings and integers, I could supply actual Domain Objects
in Fit cells. What I needed to do was provide a way of turning my objects
into strings, and parsing them from strings. Easy enough.
It didn't work - and couldn't work.
The problem is that Fitnesse never calls the custom test code provided in
the developer's own classes - at least not until you press the Test button,
to actually execute the tests. Before that point, all it does is render its
specialized Wiki markup to tables, rows and cells. If I wanted to spice up
the rendering before test execution, I had to make changes to the Wiki
engine.
It turned out that Fitnesse does provide a way of extending its Wiki markup
formats; subclassing (yuck) a WikiWidget class and hinting at its name in
a "plugins.properties" file will do the trick. So, I went about providing
a WikiWidget to do just that.
That's when I ran into the next problem.
It is natural for my Go board representations to span multiple lines: you
see a Go board even when you look at the ASCII. Rendering it to an image
is a bonus, but not necessary. The problem is that the table markup format
in Fitnesse doesn't at all deal well with cell data that spans multiple
lines. Each table row must fit in one line; if it doesn't, the rest of the
line is ignored and the Wiki displays it as if it wasn't part of the table
at all.
I looked around for a fix. I browsed the Fitnesse source code, the Fit source
code, and the tests for both. I noticed something interesting while I was
there: there is quite a lot of code to Fit and Fitnesse, but it's still
fairly easy to navigate. Some of that is due to the code being well-factored;
some of that is due to the tests providing convenient entry points, close to
the code, for such explorations; and some of that is due to the splendid
and ever improving navigation capabilities in modern IDEs. (Today I was
using Eclipse, but I'm sure IDEA would have been fine.)
Still, what emerged at the end of the day was that I was basically screwed,
short of opening up the Fitness code itself for some radical surgery.
The only thing, apparently, that can span multiple lines is a markup format
for "literal" text (not interpreted at all by the Wiki engine); this is
handled by a WikiWidget called PreProcessorLiteralWidget.
Now, by that stage I was no longer doodling but frankly hacking. So, with
little remorse, and safe in the knowledge that I'd just toss the whole thing
when I was done, I copied and pasted, wholesale, the code of PreProcessorLiteralWidget
into my own plugin widget.
It didn't work. (It couldn't work.)
It turns out that PreProcessorLiteralWidget doesn't pull this trick by its
lone self. It gets an assist from the rendering engine, which grants it the
special status of being the only widget capable of processing "literals".
There was a slight sense of outrage at this. From the outside, the "built in"
plugins appear to be just the same, and to be treated just the same, as the
ones written by Fitnesse extension developers. And perhaps they are, all of
them - but for one exception I'm now aware of, PreProcessorLiteralWidget.
So, I sat back and took stock. Fitnesse is a Wiki specialized for acceptance
testing, which wraps the testing tool Fit, whose strengths are complementary
to those of tools geared to script-style tests. I seem to be working toward
script-style tests, written in an improvised markup format which conflicts
with what Fitnesse uses. No big deal, just a missed encounter - and I now know more about Fit and Fitnesse.
Still, I'm feeling just a little bit comforted in one of my prejudices
about acceptance testing: it's better to start with the tests (even doodled
tests) than to start with the testing tool. Let your tests tell you whether
the tool is suitable - not the other way round.
Previous Topic
