Bertrand Meyer talks with Bill Venners about how Design by Contract relates to test-driven development.

Read this Artima.com interview with the creator of Eiffel:

http://www.artima.com/intv/contest.html

What do you think of Bertrand's comments?

I'd like to find out a bit more about contracts and DbC. What (freeware) resources are there, does anyone know? Or does further investigation lead inevitably to investing significant sums of money, which I don't have, and my employer probably won't spend?

Pattern-chaser

"Who cares, wins"

You can derive the specific from the general. You cannot really infer the general from the specific.

I don't know about "infer," but one of my favorite techniques, triangulation, certainly involves deriving the general from the specific. I wrote something up about it that appears here:

http://perlmonks.thepen.com/92632.html

I think this is closer to the test-driven than the contract approach, as I understand them, and I like this better.

Bill Venners,

I tried to send this message personally to you using webmaster@artima.com, but the conformation fails each time (I've tried several email addresses).
What I wanted to say is that on http://www.artima.com/intv/contest.html you write two times "MeyerS discusses", it should read "Meyer discusses".

Could you delete this message when you've read it?

> I'd like to find out a bit more about contracts and DbC.
> What (freeware) resources are there, does anyone know? Or
> does further investigation lead inevitably to investing
> significant sums of money, which I don't have, and my
> employer probably won't spend?

There's an open source GNU Eiffel compiler called SmartEiffel, which can be found at http://smarteiffel.loria.fr/.

Alternatively, I believe Eiffel Software (Bertrand Meyer's company) has a free version that can be downloaded. Their website is at http://www.eiffel.com.

I'm not too sure about DbC for other languages, although I have heard about implementations for Java and C++. A search on google may/may not bring something up.

Cheers.

Phil.

I have used DbC in a patchy (and growing) manner (in Java and C++ using my own framework), and have found it great for discovering the interface, driving the abstractions, along with proper method documentation.

I have come across, and previously done myself, lots of classes just thrown together with inappropriate interfaces; using DbC puts the brakes on.

Using JUnit is another, complimentary, way of driving the process, writing tests of how you see the class(es) being used. I think the 2 go hand-in-hand to creating great, intuitive interfaces quickly.

The problem I have is why write unit tests (after the discovery phase) for specifics when the contracts are written to catch everything. Do you trust the DbC mechanism or not? Unit tests are useful at higher levels to catch errors in aggregation.

Static analysis could catch some of these problems (e.g. the contract for method "foo" says parameter "x" must be between 1 and 100, but a user of foo writes anObject.foo( 102 ), i.e. this could be caught at compile-time). This could catch lots of errors, but probably not all (depends upon quality of contracts).

I generally agree with the views of Bertrand Meyer. In the article, I do agree with that it would be good to see some cross-pollination between the leaders in XP and design by contract. This sentiment seemed to come up frequently:

Tests are a way to find errors. You may have removed enough errors that your tests pass, and therefore you think you're OK

Like many others who are exceptional in their own field, but not knowledgeable about TDD, Meyer makes the fundamental mistake: Test-Driven Development is a design technique, not a testing technique.

Finally, I would like to pose a challenge to DbC (which I admittedly have only superficial understanding of):

In many cases, it seems to me that the contract would be as complex as the implementation. Could you outline how you would solve, say Robert C Martin's bowling example (http://www.objectmentor.com/resources/articles/xpepisode.htm) - or a bowling scorer in general, using DbC?


Thanks,
~Johannes

Like many others who are exceptional in their own field, but not knowledgeable about TDD, Meyer makes the fundamental mistake: Test-Driven Development is a design technique, not a testing technique

Could it be that TDD is fundamentally misleading?
All that talk of tests and yet TDD isn't a testing technique.

Could you outline how you would solve, say Robert C Martin's bowling example ... or a bowling scorer in general, using DbC?

Google comp.object "bowling game dbc"

And there are several incremental implementations in this thread:
http://groups.google.com/groups?q=g:thl3906003427d&dq=&hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=l11irvgd06vp6aqups002oobumiun3u3u7%404ax.com

Like many others who are exceptional in their own field, but not knowledgeable about TDD, Meyer makes the fundamental mistake: Test-Driven Development is a design technique, not a testing technique

Could it be that TDD is fundamentally misleading?
All that talk of tests and yet TDD isn't a testing technique.

Be fair, guys! "test-driven" is an adjective and "design" is the noun it describes. Would you wish to remove all mention of testing, to ensure that no-one misunderstood? ;-)

More seriously, the difference between DbC and TDD, and the ways in which they complement one another, is interesting. [Sadly I've never had the opportunity to practice either technique. Perhaps one day. Sigh. ;-)

Pattern-chaser

"Who cares, wins"]

> The word "formal" is scary to many people, so we shouldn't
> overplay the formality of Design by Contract.

This is a pity. I was actually quite dissapointed not to
find a real compile-time-verifiable DbC in Eiffel.

Of course run-time and "informal" DbC is better than
nothing. It is a certain improvement over assert()
and JavaDoc style comments but not a radical one.
"Informal" contracts can get out of sync with the
code just like function comments often do. And the
fact that you didn't get any contract violation in
the test phase does not guarantee it won't happen
in real life e.g. on Mars.

There are many areas where exaustive unit and
integration tests are simply not feasible. Mars
rover is one example. Reconfigurable system
on a chip (CPU+FPGA) is another. Here you design,
debug and verify your H/W and S/W simultaneously.
H/W simulators are notoriously slow (they have to
simulate massively parallel H/W on a sequential
CPU). There is no way you can run exaustive
tests on it. Formal verification is a big issue
there.

I also found pre/postconditions in their present form
hopelessly inadequate when describing a system of
interacting classes where the state is distributed.
The formalism simply isn't powerful enough.

If "formal" sounds scary to someone that doesn't mean
we should stop trying. After all, ADTs and function
prototypes enforce their contracts at compile time
and no-one is afraid of them now.

Regards,
Dmitri

If Eiffel doesn't support compile-time verification of DbC, then what good is DbC at all? Conceptually (even in completely unstructured code) we always have some idea of what a code block expects as inputs and produces as outputs. At some level *all* programming is designed "by contract". Without any kind of static verification of contracts, though, you need a run-time verification (read: "unit test", AKA "TDD") before you can really know whether your contract is being followed and what the consequences of not following it are.

Having a clearer idea of precisely what the contracts are is always worth something, but it comes with a cost. Is it more worthwhile to spend longer coming up with torturous unit tests for an important component or spend longer abstractly considering the contracts for a minor one? It's certainly best to have time for both, but this is not the reality that most of us live in.

I don't see anything in this article or the commentary that implies to me that DbC will in any way reduce the need for TDD. For example, this is just good sense when writing tests: "we know we can't have an exhaustive test, but we can have systematic tests that have a likelihood of exercising the cases that will fail". How does that lead to "contracts help a lot generating such "systematic" tests"? It seems that considering the contract would lead one to specify only correct inputs which would test only the simplest positive-path logic. I would consider my approach closer to "design by considering the most malicious possible run-time behavior of a user" ;-)

It seems to me that programming by contract and TDD are complimentary. I use both. If you have a test that runs a piece of source and checks for valid behavior/output, its just a bonus to have the assertions in the source to catch more errors than you might have with just the test. As far as blindly testing without any forethought goes, I don't think that either Bertrand or Kent believe this is correct. I may interpret incorrectly but I always think of what Kent is saying as "don't dwell on your thoughts too long because you're not going to come up with the right answer without writing some code first anyway" This is different than not having any forethought at all. I also believe that most time spent in creating assertions is time well spent. But how often do we dwell on assertions. If an integer must always be greater than zero, how long must we dwell on our assertion? Just code it. Assertions save my butt on every project. Test Driven development allows me to do most of my testing before I ever install my first production instance and to make modifications quickly before and after that without fear. I think we should all buy both Bertand and Kent a beer.

If Eiffel doesn't support compile-time verification of DbC, then what good is DbC at all?
None of the languages I can think of at-the-moment provide compile-time verification of array bounds in-the-general-case, nor compile-time verification of the absence of integer overflow.

Nonetheless, we seem to have monthly buffer overflow patches as a reminder that some languages don't provide the run-time bounds checking that Hoare considered essential 40 years ago.

And most languages (not Java) provide run-time integer overflow checks rather than continue silently in error.

So what good is run-time contract checking? We trap the error instead of allowing data and program to be silently corrupted.


you need a run-time verification (read: "unit test", AKA "TDD")
Firstly, "unit tests" are only present at test-time, assertions are present every-time the program is run.

Secondly, TDD is said to be a design technique not a testing technique; unit testing is a testing technique - so TDD has "programmer tests".

Test-first forces you to specify the postconditions before you code.

I usually check the preconditions on entry to a method, especially in constructors.

And I put the postconditions in the unit tests.

> None of the languages I can think of at-the-moment provide
> compile-time verification of array bounds
> in-the-general-case, nor compile-time verification of the
> absence of integer overflow.

> Nonetheless, we seem to have monthly buffer overflow
> patches as a reminder that some languages don't provide
> the run-time bounds checking that Hoare considered
> essential 40 years ago.

This is very interesting. I didn't get it from the article at all, but it seems like what you're saying is that the purpose of DbC is to provide the fastest possible failure path in the event of an error invoking a contract, and to note all the conditions which should cause something to fail.

If this is the case, then we are in violent agreement about what's necessary - as I was saying before, unit tests must verify that the program responds to invalid inputs correctly.

> Firstly, "unit tests" are only present at test-time,
> assertions are present every-time the program is run.

> Secondly, TDD is said to be a design technique not a
> testing technique; unit testing is a testing technique -
> so TDD has "programmer tests".

So I guess from my test-infected perspective, I use TDD to arrive at correct contracts more quickly. Is the fundamental technique of "design by contract" inserting assertions? Is there ever a time when these assertions can cause performance concerns (and are those concerns usually a bad idea to pay attention to, like spurious objections to garbage collectors, et. al.)?

The thing that still eludes me is what technique DbC proposes we use for arriving at which pre/postconditions are critical and which are extraneous. I understand the logic behind the TDD approach to locating these, but in the absence of postulating a test which fails and which an assertion could describe the failure mode of before the code executed, what other testable analysis techniques can be employed?

Also, what do you mean by "programmer tests"?

Thank you for these corrections, they were enlightening.