A Conversation with Andy Hunt and Dave Thomas, Part IV

---

Andy Hunt and Dave Thomas are the Pragmatic Programmers, recognized internationally as experts in the development of high-quality software. Their best-selling book of software best practices, The Pragmatic Programmer: From Journeyman to Master (Addison-Wesley, 1999), is filled with practical advice on a wide range of software development issues. They also authored Programming Ruby: A Pragmatic Programmer's Guide (Addison-Wesley, 2000), and helped to write the now famous Agile Manifesto.

In this interview, which is being published in ten weekly installments, Andy Hunt and Dave Thomas discuss many aspects of software development:

• In Part I. Don't Live with Broken Windows, they discuss the importance of software craftsmanship and the importance of staying on top of the small problems in your projects.
• In Part II. Orthogonality and the DRY Principle, they discuss the importance of keeping your system orthogonal, and the real meaning the DRY, or Don't Repeat Yourself, principle.
• In Part III. Good Enough Software, they discuss the myth of bug-free software, the importance of specifying level of quality as a system requirement, and the need for every team member to inject quality throughout the development cycle.
• In this installment, they discuss an approach to design in which details are pulled out of the code and stored as metadata.

Put Abstractions in Code, Details in Metadata

Bill Venners: In your book, The Pragmatic Programmer, you say, "Put abstractions in code, details in metadata." What do you mean by that?

Andy Hunt: Abstractions live longer than details. Details are volatile. Because details are going to change, put the details where changing them will create the least amount of friction. Typically that's outside the code base—in a database, a properties file, or XML—something acting as metadata. We suggest an overall architecture where you put your energy into creating the right abstractions in the code. And as much as possible, push the details out somewhere else where they are easier to change, because that is what's going to change.

Our philosophy contrasts with eXtreme Programming's (XP) attitude of trying to not look very far ahead, to just do the absolute minimum required at any point in time. We think where you put the details is a matter of investment. You're taking some risk when you make details easy to change that it will wind up not being necessary. But for me personally, almost every time I've taken that extra care to make a system flexible, it has saved me. So we advocate this approach very strongly in the book.

You get a feel for knowing what's going to change. The user comes in and says, "It's going to be blue. No, no, green. Nope, it's going to be blue." If they've already changed their minds four times, odds are they're not done yet. You know that's going to be volatile. Anything dealing with legal requirements is more than likely going to be volatile, so move that out where it will be easy to change.

Dave Thomas: When people read our advice in the book about metadata , they tend to imagine very complicated architectures with lots of abstraction. But in reality, it could be very simple. If the sales tax rate is currently 7%, I don't put 7% into the code. I put it into a properties file or the database. The sales tax rate is a detail I abstract out of the code and store externally.

Once you're comfortable with pulling out simple metadata, like a sales tax rate, you can go further. You can start asking yourself, "What are the actual chunks of code that I'm processing to handle this particular order of business?" For example, I implemented an online order entry system in which the processing was spread out over a number of months, depending on how payments were received. I implemented it as a set of individual business functions controlled by a state machine. I defined all the states and state transitions in a database. My abstractions were individual chunks of functionality. Metadata in the database controlled how they worked together.

That architecture saved me, because the customer kept coming back saying, "When a purchase order arrives, we actually need to do this but not that." Rather than change six pieces of code to make the system act differently, I only had to update the state machine in the database, and everything just worked. Pulling details out into metadata doesn't need to be complicated. It doesn't necessarily represent more work. It just requires a different kind of work.

Knowing What Will Change

Bill Venners: That sounds nice in theory, but how do I know which things to make configurable? How do I know which things are likely to change?

Andy Hunt: I think knowing what to make configurable is largely a matter of experience. Some of it is just common sense, but most of it is experience. In a couple of instances I have decided off the top of my head to stick a parameter into a properties file, to help make the system soft and flexible. But when I talked to the user, I learned that the parameter hadn't changed in 30 years. A parameter like that isn't likely to change in the future, so it probably belongs in the code.

Dave Thomas: That is crucial. I think another difference between us and the XP folks is that we value experience. We think your experience is a key component of your everyday work. It is perfectly valid to look at your experience and say, "The last time I did this I got bitten by this particular field changing 17 times, so this time I'm going to abstract it out into metadata."

You have to accept the fact that you're not going to get it right the first time. And you're not going to get it perfectly right the second or third time. You'll never get it perfectly right, but along the way you can get better and better . To do that, you have to discipline yourself to apply a reflective process to what you do.

Bill Venners: What do you mean by reflective process?

Dave Thomas: You always have to look back at what you did and ask, "How did I do that? Could I have done it better? Did I have to do it at all?" Get into the habit of doing that with everything you do. That way, you're consciously forcing yourself to reevaluate the way you do things.

In the 70s, there was an "inner tennis" craze. The tennis coach would put a chair in the middle of the court. You would swing the racquet and fire balls over the net towards the chair. You didn't aim for the chair. You just hit the ball and noticed whether the ball landed to the left, right, in front or behind the chair. Eventually the feedback mechanism involved would teach you that when you do this, the ball goes over there. When you do this, the ball goes over here.

Andy Hunt: So the chair was just providing a reference point.

Dave Thomas: Exactly. And it's the same with a project. If you give yourself constant feedback, then gradually you'll get better at doing what you do. That's absolutely critical, but most people don't do that. They rush onto the next project, without going back and thinking how they did this last one. Without that feedback, you're never going to improve.

Andy Hunt: Another point is that we hear a lot about agile methodologies these days, but not a lot about writing agile code. If you want to be able to keep up with rapid changes on a project, however, you have to make the code agile. You have to be able to make changes quickly. The XP folks say the way to do that is via refactoring. They recommend you always keep the code tidy and well factored enough that you can make needed changes fairly quickly. You can take the XP approach of refactoring the code, but if pull details out of the code into metadata, you can make changes without even having to touch the code.

In addition, with metadata you have the added benefit that you can make changes out in the field to a system that's already been deployed. If a customer calls and tells you their MP3 player isn't working, you may be able to tell them to switch a parameter in a property file. The MP3 player will use a different decoder and algorithm and get around the bug. So the more metadata you have, the more flexibility you have. And flexibility translates into being agile.

Dave Thomas: In addition, once you start planning your code this way, you find that the discipline actually improves the design of the code.

Bill Venners: The discipline of separating abstraction and details?

Dave Thomas: Yes, once you adopt a discipline of separating the abstraction and the data that abstraction is working on, your code starts being structured in a far tidier way. Your code tends to be decoupled and orthogonal. The discipline is to structure your code in such a way that you can drive it from metadata. By doing that you're guaranteed to have nice, self-contained, clean chunks of functionality that you can assemble in different ways.

Configuration versus Coding

Bill Venners: Why is configuring with data or a mini-language better than coding?

Andy Hunt: You are trying to set up an environment where you can more directly, concisely, and clearly express what the user wants. When there's one change in the real world, you want to be able to make only one change in the code. By setting up an environment that includes a dedicated domain language, you come much closer to that goal. When something in the real world changes, you just change one high level statement. If the entire system were coded in a regular programming language—Java, C++, Ruby, whatever—you might have to change a dozen lines of code. If you have a special purpose language or some configuration data more closely tied to the real world, poof, you can just change one parameter. Metadata you helps you avoid the multiplicative effect real world changes can have on the code base.

Next Week

Come back Monday, March 31 for Part V of this conversation with Pragmatic Programmers Andy Hunt and Dave Thomas. If you'd like to receive a brief weekly email announcing new articles at Artima.com, please subscribe to the Artima Newsletter.

Resources

Andy Hunt and Dave Thomas are authors of The Pragmatic Programmer, which is available on Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/020161622X/

The Pragmatic Programmer's home page is here:
http://www.pragmaticprogrammer.com/

Dave Thomas was not the first person I've interviewed who mentioned the arcade game Whack-a-Mole. James Gosling also called upon the versatile Whack-a-Mole metaphor while pointing out that it is sometimes hard in engineering to know if you've solved a problem or moved it:
http://www.artima.com/intv/gosling34.html

The Agile Manifesto is here:
http://agilemanifesto.org/

Ward's Wiki, the first WikiWikiWeb, created by Ward Cunningham, is here:
http://c2.com/cgi/wiki?WelcomeVisitors

Extreme Programming: A Gentle Introduction:
http://www.extremeprogramming.org/

XProgramming.com: An Extreme Programming Resource:
http://www.xprogramming.com/

Talk back!

Have an opinion? Readers have already posted 6 comments about this article. Why not add yours?

About the author

Bill Venners is president of Artima Software, Inc. and editor-in-chief of Artima.com. He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project that produced the ServiceUI API. The ServiceUI became the de facto standard way to associate user interfaces to Jini services, and was the first Jini community standard approved via the Jini Decision Process. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community. He currently devotes most of his energy to building Artima.com into an ever more useful resource for developers.