Article Discussion
Contract-Driven Development
Summary: Bertrand Meyer talks with Bill Venners about how Design by Contract relates to test-driven development.
22 posts.
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: May 24, 2004 2:08 AM by mudit
    Bill
     
    Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
    Contract-Driven Development
    February 29, 2004 9:00 PM      
    Bertrand Meyer talks with Bill Venners about how Design by Contract relates to test-driven development.

    Read this Artima.com interview with the creator of Eiffel:

    http://www.artima.com/intv/contest.html

    What do you think of Bertrand's comments?
    • Bert
       
      Posts: 1 / Nickname: dichtertje / Registered: July 30, 2003 8:55 AM
      Re: Contract-Driven Development
      March 2, 2004 2:23 AM      
      Bill Venners,

      I tried to send this message personally to you using webmaster@artima.com, but the conformation fails each time (I've tried several email addresses).
      What I wanted to say is that on http://www.artima.com/intv/contest.html you write two times "MeyerS discusses", it should read "Meyer discusses".

      Could you delete this message when you've read it?
    • Steve
       
      Posts: 2 / Nickname: patterncha / Registered: January 20, 2004 8:18 PM
      Re: Contract-Driven Development
      March 2, 2004 2:02 AM      
      I'd like to find out a bit more about contracts and DbC. What (freeware) resources are there, does anyone know? Or does further investigation lead inevitably to investing significant sums of money, which I don't have, and my employer probably won't spend?

      Pattern-chaser

      "Who cares, wins"
      • Phil
         
        Posts: 1 / Nickname: philmalin / Registered: March 1, 2004 10:50 PM
        Re: Contract-Driven Development
        March 2, 2004 3:54 AM      
        > I'd like to find out a bit more about contracts and DbC.
        > What (freeware) resources are there, does anyone know? Or
        > does further investigation lead inevitably to investing
        > significant sums of money, which I don't have, and my
        > employer probably won't spend?

        There's an open source GNU Eiffel compiler called SmartEiffel, which can be found at http://smarteiffel.loria.fr/.

        Alternatively, I believe Eiffel Software (Bertrand Meyer's company) has a free version that can be downloaded. Their website is at http://www.eiffel.com.

        I'm not too sure about DbC for other languages, although I have heard about implementations for Java and C++. A search on google may/may not bring something up.

        Cheers.

        Phil.
    • David
       
      Posts: 1 / Nickname: dicky / Registered: March 1, 2004 10:40 PM
      Re: Contract-Driven Development
      March 2, 2004 3:59 AM      
      I have used DbC in a patchy (and growing) manner (in Java and C++ using my own framework), and have found it great for discovering the interface, driving the abstractions, along with proper method documentation.

      I have come across, and previously done myself, lots of classes just thrown together with inappropriate interfaces; using DbC puts the brakes on.

      Using JUnit is another, complimentary, way of driving the process, writing tests of how you see the class(es) being used. I think the 2 go hand-in-hand to creating great, intuitive interfaces quickly.

      The problem I have is why write unit tests (after the discovery phase) for specifics when the contracts are written to catch everything. Do you trust the DbC mechanism or not? Unit tests are useful at higher levels to catch errors in aggregation.

      Static analysis could catch some of these problems (e.g. the contract for method "foo" says parameter "x" must be between 1 and 100, but a user of foo writes anObject.foo( 102 ), i.e. this could be caught at compile-time). This could catch lots of errors, but probably not all (depends upon quality of contracts).
    • Johannes
       
      Posts: 3 / Nickname: jhannes / Registered: June 29, 2003 7:10 AM
      Re: Contract-Driven Development
      March 2, 2004 4:01 AM      
      I generally agree with the views of Bertrand Meyer. In the article, I do agree with that it would be good to see some cross-pollination between the leaders in XP and design by contract. This sentiment seemed to come up frequently:

      Tests are a way to find errors. You may have removed enough errors that your tests pass, and therefore you think you're OK

      Like many others who are exceptional in their own field, but not knowledgeable about TDD, Meyer makes the fundamental mistake: Test-Driven Development is a design technique, not a testing technique.

      Finally, I would like to pose a challenge to DbC (which I admittedly have only superficial understanding of):

      In many cases, it seems to me that the contract would be as complex as the implementation. Could you outline how you would solve, say Robert C Martin's bowling example (http://www.objectmentor.com/resources/articles/xpepisode.htm) - or a bowling scorer in general, using DbC?


      Thanks,
      ~Johannes
      • Isaac
         
        Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
        Re: Contract-Driven Development
        March 2, 2004 8:30 AM      
        Like many others who are exceptional in their own field, but not knowledgeable about TDD, Meyer makes the fundamental mistake: Test-Driven Development is a design technique, not a testing technique

        Could it be that TDD is fundamentally misleading?
        All that talk of tests and yet TDD isn't a testing technique.

        Could you outline how you would solve, say Robert C Martin's bowling example ... or a bowling scorer in general, using DbC?

        Google comp.object "bowling game dbc"

        And there are several incremental implementations in this thread:
        http://groups.google.com/groups?q=g:thl3906003427d&dq=&hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=l11irvgd06vp6aqups002oobumiun3u3u7%404ax.com
        • Steve
           
          Posts: 2 / Nickname: patterncha / Registered: January 20, 2004 8:18 PM
          Re: Contract-Driven Development
          March 3, 2004 1:22 AM      
          Like many others who are exceptional in their own field, but not knowledgeable about TDD, Meyer makes the fundamental mistake: Test-Driven Development is a design technique, not a testing technique

          Could it be that TDD is fundamentally misleading?
          All that talk of tests and yet TDD isn't a testing technique.


          Be fair, guys! "test-driven" is an adjective and "design" is the noun it describes. Would you wish to remove all mention of testing, to ensure that no-one misunderstood? ;-)

          More seriously, the difference between DbC and TDD, and the ways in which they complement one another, is interesting. [Sadly I've never had the opportunity to practice either technique. Perhaps one day. Sigh. ;-)

          Pattern-chaser

          "Who cares, wins"]
    • Mark
       
      Posts: 4 / Nickname: markm / Registered: November 13, 2003 6:14 PM
      Re: Contract-Driven Development
      March 5, 2004 1:20 PM      
      It seems to me that programming by contract and TDD are complimentary. I use both. If you have a test that runs a piece of source and checks for valid behavior/output, its just a bonus to have the assertions in the source to catch more errors than you might have with just the test. As far as blindly testing without any forethought goes, I don't think that either Bertrand or Kent believe this is correct. I may interpret incorrectly but I always think of what Kent is saying as "don't dwell on your thoughts too long because you're not going to come up with the right answer without writing some code first anyway" This is different than not having any forethought at all. I also believe that most time spent in creating assertions is time well spent. But how often do we dwell on assertions. If an integer must always be greater than zero, how long must we dwell on our assertion? Just code it. Assertions save my butt on every project. Test Driven development allows me to do most of my testing before I ever install my first production instance and to make modifications quickly before and after that without fear. I think we should all buy both Bertand and Kent a beer.
    • Kent
       
      Posts: 1 / Nickname: windchill / Registered: March 5, 2004 10:52 AM
      Re: Contract-Driven Development
      March 5, 2004 3:56 PM      
      Test-first forces you to specify the postconditions before you code.

      I usually check the preconditions on entry to a method, especially in constructors.

      And I put the postconditions in the unit tests.
      • Frank
         
        Posts: 11 / Nickname: fmitchell / Registered: January 31, 2003 9:53 AM
        Re: Contract-Driven Development
        March 5, 2004 7:48 PM      
        For an interesting essay on the relationship between TDD and DbC, see http://www.onestepback.org/index.cgi/Tech/Programming/DbcAndTesting.html

        Note also the postscript, where the author admits that he now uses TDD almost exclusively ...
        • Isaac
           
          Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
          Re: Contract-Driven Development
          March 6, 2004 9:30 AM      
          For an interesting essay on the relationship between TDD and DbC
          Thanks

          where the author admits that he now uses TDD almost exclusively...
          Perhaps you imply more than the author states ;-)
          Maybe the languages he's now using have little support for DbC and good support for TDD?
          • Frank
             
            Posts: 11 / Nickname: fmitchell / Registered: January 31, 2003 9:53 AM
            Re: Contract-Driven Development
            March 6, 2004 9:11 PM      
            > Perhaps you imply more than the author states ;-)
            > Maybe the languages he's now using have little support for
            > DbC and good support for TDD?

            That's probably the main reason. As far as I can tell, to do DbC right, it needs to be an integral part of the language you're working in ... otherwise, it's really hard to do preconditions on abstract ("deferred") classes and routines, inheritance of preconditions to preserve Liskov substitutablilty, and automatic documentation of preconditions. This means a) the language has DbC from the start (only Eiffel and Sather, AFAIK), b) the language allows metaobject protocols and hygenic macros (e.g. Smalltalk, Lisp, and similar languages), or c) the programmer introduces a preprocessor (e.g. iContract or JContract for JUnit).

            Implementing DbC with macros, metaobject protocols, or preprocessors is probably a lot of work. Aspects may be easier, although I've never liked the idea of contracts in a separate file from the methods they describe; there's too much opportunity for obsolete contracts, and enough context-switching to discourage people from writing contracts unless they have to.

            OTOH, unit testing is almost trivial to implement, and covers enough of the same territory. What unit testing lacks is the ability to trap and diagnose problems during acceptance tests and production operation. (Also, implementing "abstract tests" for arbitrary descendants is a real pain, and unit-testing doesn't have the auto-documentation ability of DbC.)
            • Frank
               
              Posts: 11 / Nickname: fmitchell / Registered: January 31, 2003 9:53 AM
              Re: Contract-Driven Development
              March 6, 2004 9:13 PM      
              > ... iContract or JContract for JUnit ...

              s/JUnit/Java/

              Maybe I need a professional proofreader.
    • DChaplin
       
      Posts: 1 / Nickname: dchaplin / Registered: April 17, 2004 0:01 AM
      Re: Contract-Driven Development
      April 17, 2004 4:12 AM      
      You might like to read my paper Contractual Test-Driven Development which I wrote in April 2003 where I explain effective methods to combine TDD and DBC.

      http://www.byte-vision.com/CtddArticle.aspx

      This was based on a real life project project where I coached and led an XP team for 18 months and we succesfully used both techniques together.

      We found DBC and TDD pulled in different directions, but finally found a common ground that was very effective.

      It was particularly interesting to actually do the real thing to reach the conclusions after our original theories were blown out of the water when we first tried them.

      Suffice to say, there nothing like a good theory being born out of practical experience.

      Regards

      Dave Chaplin
    • Carl
       
      Posts: 3 / Nickname: cmanaster / Registered: June 3, 2003 3:30 AM
      Re: Contract-Driven Development
      March 2, 2004 2:17 AM      
      You can derive the specific from the general. You cannot really infer the general from the specific.

      I don't know about "infer," but one of my favorite techniques, triangulation, certainly involves deriving the general from the specific. I wrote something up about it that appears here:

      http://perlmonks.thepen.com/92632.html

      I think this is closer to the test-driven than the contract approach, as I understand them, and I like this better.
      • mudit
         
        Posts: 3 / Nickname: mbhandari / Registered: February 22, 2004 5:43 AM
        Re: Contract-Driven Development
        May 24, 2004 1:07 AM      
        Contract-Driven Development is fine
    • Dmitri
       
      Posts: 1 / Nickname: dmitrik / Registered: March 3, 2004 9:53 PM
      Re: Contract-Driven Development
      March 4, 2004 7:52 AM      
      > The word "formal" is scary to many people, so we shouldn't
      > overplay the formality of Design by Contract.

      This is a pity. I was actually quite dissapointed not to
      find a real compile-time-verifiable DbC in Eiffel.

      Of course run-time and "informal" DbC is better than
      nothing. It is a certain improvement over assert()
      and JavaDoc style comments but not a radical one.
      "Informal" contracts can get out of sync with the
      code just like function comments often do. And the
      fact that you didn't get any contract violation in
      the test phase does not guarantee it won't happen
      in real life e.g. on Mars.

      There are many areas where exaustive unit and
      integration tests are simply not feasible. Mars
      rover is one example. Reconfigurable system
      on a chip (CPU+FPGA) is another. Here you design,
      debug and verify your H/W and S/W simultaneously.
      H/W simulators are notoriously slow (they have to
      simulate massively parallel H/W on a sequential
      CPU). There is no way you can run exaustive
      tests on it. Formal verification is a big issue
      there.

      I also found pre/postconditions in their present form
      hopelessly inadequate when describing a system of
      interacting classes where the state is distributed.
      The formalism simply isn't powerful enough.

      If "formal" sounds scary to someone that doesn't mean
      we should stop trying. After all, ADTs and function
      prototypes enforce their contracts at compile time
      and no-one is afraid of them now.

      Regards,
      Dmitri
      • Glyph
         
        Posts: 2 / Nickname: glyph / Registered: November 17, 2003 3:52 AM
        Re: Contract-Driven Development
        March 5, 2004 0:41 PM      
        If Eiffel doesn't support compile-time verification of DbC, then what good is DbC at all? Conceptually (even in completely unstructured code) we always have some idea of what a code block expects as inputs and produces as outputs. At some level *all* programming is designed "by contract". Without any kind of static verification of contracts, though, you need a run-time verification (read: "unit test", AKA "TDD") before you can really know whether your contract is being followed and what the consequences of not following it are.

        Having a clearer idea of precisely what the contracts are is always worth something, but it comes with a cost. Is it more worthwhile to spend longer coming up with torturous unit tests for an important component or spend longer abstractly considering the contracts for a minor one? It's certainly best to have time for both, but this is not the reality that most of us live in.

        I don't see anything in this article or the commentary that implies to me that DbC will in any way reduce the need for TDD. For example, this is just good sense when writing tests: "we know we can't have an exhaustive test, but we can have systematic tests that have a likelihood of exercising the cases that will fail". How does that lead to "contracts help a lot generating such "systematic" tests"? It seems that considering the contract would lead one to specify only correct inputs which would test only the simplest positive-path logic. I would consider my approach closer to "design by considering the most malicious possible run-time behavior of a user" ;-)
        • Isaac
           
          Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
          Re: Contract-Driven Development
          March 5, 2004 2:43 PM      
          If Eiffel doesn't support compile-time verification of DbC, then what good is DbC at all?
          None of the languages I can think of at-the-moment provide compile-time verification of array bounds in-the-general-case, nor compile-time verification of the absence of integer overflow.

          Nonetheless, we seem to have monthly buffer overflow patches as a reminder that some languages don't provide the run-time bounds checking that Hoare considered essential 40 years ago.

          And most languages (not Java) provide run-time integer overflow checks rather than continue silently in error.

          So what good is run-time contract checking? We trap the error instead of allowing data and program to be silently corrupted.


          you need a run-time verification (read: "unit test", AKA "TDD")
          Firstly, "unit tests" are only present at test-time, assertions are present every-time the program is run.

          Secondly, TDD is said to be a design technique not a testing technique; unit testing is a testing technique - so TDD has "programmer tests".
          • Glyph
             
            Posts: 2 / Nickname: glyph / Registered: November 17, 2003 3:52 AM
            Re: Contract-Driven Development
            March 5, 2004 7:46 PM      
            > None of the languages I can think of at-the-moment provide
            > compile-time verification of array bounds
            > in-the-general-case, nor compile-time verification of the
            > absence of integer overflow.

            > Nonetheless, we seem to have monthly buffer overflow
            > patches as a reminder that some languages don't provide
            > the run-time bounds checking that Hoare considered
            > essential 40 years ago.

            This is very interesting. I didn't get it from the article at all, but it seems like what you're saying is that the purpose of DbC is to provide the fastest possible failure path in the event of an error invoking a contract, and to note all the conditions which should cause something to fail.

            If this is the case, then we are in violent agreement about what's necessary - as I was saying before, unit tests must verify that the program responds to invalid inputs correctly.

            > Firstly, "unit tests" are only present at test-time,
            > assertions are present every-time the program is run.

            > Secondly, TDD is said to be a design technique not a
            > testing technique; unit testing is a testing technique -
            > so TDD has "programmer tests".

            So I guess from my test-infected perspective, I use TDD to arrive at correct contracts more quickly. Is the fundamental technique of "design by contract" inserting assertions? Is there ever a time when these assertions can cause performance concerns (and are those concerns usually a bad idea to pay attention to, like spurious objections to garbage collectors, et. al.)?

            The thing that still eludes me is what technique DbC proposes we use for arriving at which pre/postconditions are critical and which are extraneous. I understand the logic behind the TDD approach to locating these, but in the absence of postulating a test which fails and which an assertion could describe the failure mode of before the code executed, what other testable analysis techniques can be employed?

            Also, what do you mean by "programmer tests"?

            Thank you for these corrections, they were enlightening.
            • Isaac
               
              Posts: 51 / Nickname: igouy / Registered: July 10, 2003 7:42 AM
              Re: Contract-Driven Development
              March 6, 2004 9:19 AM      
              it seems like what you're saying is...
              I am not particularly knowledgeable about DbC - there are far better sources of information:
              "Design By Contract with JML"
              ftp://ftp.cs.iastate.edu/pub/leavens/JML/jmldbc.pdf

              to note all the conditions which should cause something to fail
              My understanding is that the contract specifies the conditions required for success, not those which should be regarded as failure.

              Also, what do you mean by "programmer tests"?
              My understanding was that some TDD practitioners have started talking about "programmer tests" rather than "unit tests" to make the distinction that these "programmer tests" are being used to create code - not to test code.

              Thank you for these corrections, they were enlightening
              And possibly misleading ;-)
              • mudit
                 
                Posts: 3 / Nickname: mbhandari / Registered: February 22, 2004 5:43 AM
                Re: Contract-Driven Development
                May 24, 2004 2:08 AM      
                test reply