A Conversation with Andy Hunt and Dave Thomas, Part VII

---

Andy Hunt and Dave Thomas are the Pragmatic Programmers, recognized internationally as experts in the development of high-quality software. Their best-selling book of software best practices, The Pragmatic Programmer: From Journeyman to Master (Addison-Wesley, 1999), is filled with practical advice on a wide range of software development issues. They also authored Programming Ruby: A Pragmatic Programmer's Guide (Addison-Wesley, 2000), and helped to write the now famous Agile Manifesto.

In this interview, which is being published in ten weekly installments, Andy Hunt and Dave Thomas discuss many aspects of software development:

• In Part I. Don't Live with Broken Windows, they discuss the importance of software craftsmanship and the importance of staying on top of the small problems in your projects.
• In Part II. Orthogonality and the DRY Principle, they discuss the importance of keeping your system orthogonal, and the real meaning the DRY, or Don't Repeat Yourself, principle.
• In Part III. Good Enough Software, they discuss the myth of bug-free software, the importance of specifying level of quality as a system requirement, and the need for every team member to inject quality throughout the development cycle.
• In Part IV. Abstraction and Detail, they discuss an approach to design in which details are pulled out of the code and stored as metadata.
• In Part V. Building Adaptable Systems, they discuss reversible design decisions, the cost of change curve, going beyond the requirements, and making systems configurable.
• In Part VI. Programming Close to the Domain, they discuss the benefit of programming in a language close to the business domain.
• In this installment, they discuss a gardening metaphor for software development, the reasons coding is not mechanical, and the stratification of development jobs.

Software Development as Gardening

Bill Venners: In your book, The Pragmatic Programmer, you say, "Rather than construction, programming is more like gardening." I really like your gardening metaphor for software development. Can you elaborate on it?

Andy Hunt: There is a persistent notion in a lot of literature that software development should be like engineering. First, an architect draws up some great plans. Then you get a flood of warm bodies to come in and fill the chairs, bang out all the code, and you're done. A lot of people still feel that way. I saw an interview in the last six months of a big outsourcing house in India where this was how they felt. They paint a picture of constructing software like buildings. The high talent architects do the design. The coders do the constructing. The tenants move in, and everyone lives happily ever after. We don't think that's very realistic. It doesn't work that way with software.

We paint a different picture. Instead of that very neat and orderly procession, which doesn't happen even in the real world with buildings, software is much more like gardening. You do plan. You plan you're going to make a plot this big. You're going to prepare the soil. You bring in a landscape person who says to put the big plants in the back and short ones in the front. You've got a great plan, a whole design.

But when you plant the bulbs and the seeds, what happens? The garden doesn't quite come up the way you drew the picture. This plant gets a lot bigger than you thought it would. You've got to prune it. You've got to split it. You've got to move it around the garden. This big plant in the back died. You've got to dig it up and throw it into the compost pile. These colors ended up not looking like they did on the package. They don't look good next to each other. You've got to transplant this one over to the other side of the garden.

Dave Thomas: Also with a garden, there's a constant assumption of maintenance. Everybody says, I want a low maintenance garden, but the reality is a garden is something that you're always interacting with to improve or even just keep the same. Although I know there's building maintenance, you typically don't change the shape of a building. It just sits there. We want people to view software as being far more organic, far more malleable, and something that you have to be prepared to interact with to improve all the time.

Coding is not Mechanical

Bill Venners: You say in your book, "Conventional wisdom says that once the project is in the coding phase, the work is mostly mechanical, transcribing the design into executable statements. We think this attitude is the single biggest reason that many programs are ugly, inefficient, poorly structured, unmaintainable, just plain wrong. Coding is not mechanical." You seem to feel strongly about that.

Dave Thomas: The ultimate objective of writing software is to produce code that generates some business value. And that value is produced by writing code. It's only at the time that you write the code that you know, yes, this is actually working. Before that time, it's all theory. You can do your best to validate your designs and check your architectures, but really it's all theory until it actually runs. In the running of the code, you get the feedback that you need to know you're going in the right direction.

What happens if you don't do that? What happens if back at the design level you say, "I'm an omnipotent designer. I can produce things. I know they'll be right. These are just coding details." You are eliminating an entire level of feedback. And I think it's actually the most valuable feedback you can get, because it's the feedback that says, "This is what the program does."

Quite often you'll be programming some requirement that should be simple, but turns out to be really complicated. Why is this? Often it is because the requirement has been poorly stated or understood. You say, "Wait a minute, I don't like how this is turning out." You go back to the client and say, "You asked for this, but it is having all these ripple effects in the code. Why is this?" And the client will say, "Oh, I didn't really mean it that way. I meant it this way." The code is a mirror, a model of the reality you're actually implementing. If the code has problems, it could well be because the reality as expressed to you has problems. Simply seeing code as mechanical loses all the benefit of that feedback.

Andy Hunt: As an anecdote, on one project Dave and I were implementing legal requirements in code. These were actual laws that had been passed. We got about three quarters the way through it and realized that the law as stated was wrong. It was inconsistent. If you read through the law one way, it said something had to happen. But in the next paragraph it contradicted itself. Nobody had caught it up to that point, because nobody had tried to do a detailed implementation enacting this legal requirement. This mere mechanical act of coding actually catches bugs in the real world. If you dismiss it as just a mechanical translation, what are you going to do? You find genuine requirements problems while coding. You've got to be able to react

Stratification of Developer Jobs

Dave Thomas: The other dimension to this whole issue—and this is something I feel very strongly about—is the stratification of job titles. The idea that you have business analysts, architects, designers, coders, and then people considered the scum, the testers, is a really major cause of problems in the industry, for a number of reasons. First of all, you have the problem that no one feels responsible, because it is always somebody else that's handing the stuff to you.

Andy Hunt: Or somebody that you're going to pawn it off onto next.

Dave Thomas: Right. You also have a phenomenal communication overhead. Every single thing has to pass up and down the chain. Everything has to be a memo. The business analyst to the architect, the architect to the designer, designer to coder, coder to tester, tester to coder...

Andy Hunt: And you lose signal through each interface, and introduce noise.

Dave Thomas: Right. You get this children's party game of telephone, where a requirement goes in on one end. Every time it gets transferred from one level to the next, it gets subtly distorted. You start out wanting an order entry system, and you end up with a payroll system by the time it gets down to the programmers.

Bill Venners: What is the alternative to stratification of job roles?

Dave Thomas: I don't think you can get rid of the layers. I think politically that would be a mistake. The reality is that many people feel comfortable doing things at a certain level. What you can do, however, is stop making the levels an absolute. A designer throws a design over the partition to a set of coders, who scramble for them like they are pieces of bread and they are starving, and they start coding them up. The problem is this entrenched idea that designers and coders are two separate camps. Designers and coders are really just ends of the spectrum, and everybody has elements of both. Similarly with quality assurance (QA). Nobody is just a tester.

What we should be doing is creating an environment in which people get to use all their skills. Maybe as time goes on, people move through the skill set. So today, you're 80% coder 20% designer. On the next project, you're 70% coder and 30% designer. You're moving up a little bit each time, rather than suddenly discovering a letter on your desk that says, "Today you are a designer."

Andy Hunt: If the designer also has to implement what he designs, he is, as they say, getting to eat his own dog food. The next time he does a design, he's going to have had that feedback experience. He'll be in a much better position to say, "That didn't quite work the way I thought last time. I'm going to do this better. I'm going to do this differently." Feedback paths contribute to continual learning and continual improvement.

Bill Venners: Continual learning is part of the notion of software craftsmanship?

Andy Hunt: Exactly. First you've got the designer doing some coding, so she'll do a better design next time. Also, you really need to plan how you are going to take some of your better coders and make them designers. As Dave says, will you give them "the letter" one day? Sprinkle some magic fairy dust and poof, they're now a designer? That doesn't sound like a good idea. Instead you should break them in gradually. Maybe they work with that designer, do a little bit of this design today, a little bit of that design tomorrow. Some day they can take on a greater role.

Career Paths of Software Professionals

Dave Thomas: Also, typically organizations have a very hierarchical career path. If you start off as a coder, then your route upwards basically gets narrower and narrower up the tree. But that that doesn't really suit the human in us. Because the human in us says, "I don't want to spend all my time managing these people. I want to get my hands dirty every now and then." I would like to see a far more holistic approach to career, where you don't pigeonhole people into particular roles, but allow them to have the flexibility to do a little bit of this a little bit of that, just to keep themselves in the picture.

Andy Hunt: And you can see the problems that come from this division. You can see in most companies, as Dave said, the scum of the earth, the testers. There's a real division between people testing the code, when that's their full time position—"I'm a tester"—and the people writing the code—"I'm a developer". It sets up a natural antagonism between these two groups, which most of the time probably should be the same people, at least very closely aligned groups. Instead, these are like armed camps throwing stuff over the wall at each other. Anything that sets up that kind of adversarial relationship just doesn't work.

Next Week

Come back Monday, April 21 for Part VIII of this conversation with Pragmatic Programmers Andy Hunt and Dave Thomas. If you'd like to receive a brief weekly email announcing new articles at Artima.com, please subscribe to the Artima Newsletter.

Resources

Andy Hunt and Dave Thomas are authors of The Pragmatic Programmer, which is available on Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/020161622X/

The Pragmatic Programmer's home page is here:
http://www.pragmaticprogrammer.com/

Dave Thomas was not the first person I've interviewed who mentioned the arcade game Whack-a-Mole. James Gosling also called upon the versatile Whack-a-Mole metaphor while pointing out that it is sometimes hard in engineering to know if you've solved a problem or moved it:
http://www.artima.com/intv/gosling34.html

The Agile Manifesto is here:
http://agilemanifesto.org/

Ward's Wiki, the first WikiWikiWeb, created by Ward Cunningham, is here:
http://c2.com/cgi/wiki?WelcomeVisitors

Extreme Programming: A Gentle Introduction:
http://www.extremeprogramming.org/

XProgramming.com: An Extreme Programming Resource:
http://www.xprogramming.com/

Talk back!

Have an opinion? Readers have already posted 4 comments about this article. Why not add yours?

About the author

Bill Venners is president of Artima Software, Inc. and editor-in-chief of Artima.com. He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project that produced the ServiceUI API. The ServiceUI became the de facto standard way to associate user interfaces to Jini services, and was the first Jini community standard approved via the Jini Decision Process. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community. He currently devotes most of his energy to building Artima.com into an ever more useful resource for developers.