A Conversation with Erich Gamma, Part V

---

Erich Gamma lept onto the software world stage in 1995 as co-author of the best-selling book Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley, 1995) [1]. This landmark work, often referred to as the Gang of Four (GoF) book, cataloged 23 specific solutions to common design problems. In 1998, he teamed up with Kent Beck to produce JUnit [2], the de facto unit testing tool in the Java community. Gamma currently is an IBM Distinguished Engineer at IBM's Object Technology International (OTI) lab in Zurich, Switzerland. He provides leadership in the Eclipse community, and is responsible for the Java development effort for the Eclipse platform [3].

On October 27, 2004, Bill Venners met with Erich Gamma at the OOPSLA conference in Vancouver, Canada. In this interview, which is being published in multiple installments in Leading-Edge Java on Artima Developer, Gamma gives insights into software design and development. Unlike the other articles in this series, this interview installment is based on a phone interview that took place on June 16th, 2005, shortly before Eclipse version 3.1 was released.

• In Part I: How to Use Design Patterns, Gamma describes gives his opinion on the appropriate ways to think about and use design patterns, and describes the difference between patterns libraries, such as GoF, and an Alexandrian pattern language.
• In Part II: Erich Gamma on Flexibility and Reuse, Gamma discusses the importance of reusability, the risks of speculating, and the problem of frameworkitis.
• In Part III. Design Principles from Design Patterns, Gamma discusses two design principles highlighted in the GoF book: program to an interface, not an implementation, and favor object composition over class inheritance.
• In Part IV: Patterns and Practice, Gamma discusses how design patterns are problem solution pairs, how design patterns help you understand intent and tradeoffs, and how to become a better designer through practice.
• In this fifth installment, Gamma describes the development process used by the Eclipse team, the team's "culture of shipping," and the importance of transparency in building a community around your product.

A culture of shipping

Bill Venners: How long have you been involved with the Eclipse project, and what are the most significant lessons you feel you and the Eclipse team have learned about the development process through that experience?

Erich Gamma: I've been with Eclipse since the beginning. It is actually difficult to give a fixed starting date, because there were some projects that were naturally evolving into Eclipse. For example, I had already been working on the Visual Age Micro Edition integrated development environment. Some components started in this project and were continuously refined and became a part of Eclipse. It was also during this project that SWT was born. I guess it's been about six years.

When the Eclipse project began, OTI had a culture that focused on shipping on time which has always put people in the center of the development process. Since then I've been amongst others working with John Wiegand tweaking the way we develop. During the past six years we have continually adapted. We tried out things. The things that didn't work we stopped doing and things that worked we kept on doing. And, our teams made it all work.

One thing we learned was the importance of teasing out continuous deliverables. Initially we didn't have that. As a consequence we had stressful finish sprints towards the end of a release cycle. Now we have short delivery intervals. Every six weeks is a milestone. This is something we evolved to, and now we are moving to a model of always striving to be ready to ship—we always want to be in beta.

Since we are open source, we also learned the importance of the feedback loop with the community. It is vital to interact with the community and to be transparent so the community can participate. We have achieved a good transparency level, but we are still continuously striving to improve and involve the community even more. It's obvious, isn't it, that you get a better end result with more community involvement?

Bill Venners: It seems like there's a culture of shipping that you value.

Erich Gamma: That's right. In software, having cool ideas is nice, but shipping them is what counts. For us it only counts if you have shipped the thing. That's really the mindset we have. And given that you focus on shipping, we never want to be in a mode of always being two years away from shipping. You need to have a short-term deliverable. You also plan, decide and act with this mindset. You are very risk- aware— you know what you can do so you can still ship on time with quality.

Continuous deliverables

Bill Venners: How do you accomplish continuous deliverables in the Eclipse project?

Erich Gamma: We split the release cycle into milestones at a granularity of six weeks, and each milestone ends with an improved and useable Eclipse build. In general, those six weeks are like a small development cycle, in which we plan, develop, and test. With this kind of fractal major plan, we get in effect several small development cycles for each release. We slow down at the end of each milestone. We have a day where everybody gets out of the water and does testing. Doing testing for each milestone avoids that we accumulate a larger testing effort until the end of the release cycle. Then we document what's new and noteworthy, and we announce it to the community so they can observe our progress and provide early feedback. Then we plan the next milestone, taking into account both the overall plan and individual component plans.

So we always want to be in beta. In particular, we don't want to go dark, where people can't see what we're doing for months. The goal is to have our community tracking our progress and providing continuous feedback.

Bill Venners: Why is it valuable to have the community track your progress? Is transparent development something more appropriate to open source, or is it something you think closed source projects can also find beneficial?

Erich Gamma: I think this is independent of open or closed-source. Software creates communities and transparent development is important if you want to grow a community. Open source in particular, though, is not just about making source available under some license; it is really about building up a community. And you build a community by showing them what you're up to, which means you make your plans visible. All of our milestone plans and project plans are visible on the web. All of our bugs are visible. The community really sees what's going on. Of course, what we hope for in return is that the community participates. And participation can come in many different forms—for example providing feedback in bug reports, contributing newsgroup replies, providing patches, implementing additional plug-ins, or writing articles. These are the ingredients of a tight feedback loop, and this kind of feedback loop is the key to having a good, shippable product in the end. The fact that Eclipse has such an active community is really cool and a major asset. Having such a community is an asset no matter whether the environment is open-source or closed.

Bill Venners: You said you plan, you execute, and you test. How long do you plan at the beginning of one of the six-week mini-development cycles?

Erich Gamma: We usually try to have our milestone plan posted after one week. The last week in a cycle is the "chill down." Typically we do a candidate build on Tuesday, testing on Wednesday, fixing critical bugs on Thursday, and declaring the milestone build on Friday. To declare a milestone build each component team needs to say, "Yes, my component is good to go." If there are still problems with a build the component requests an additional build.

It's kind of like driving a bus, and figuring out how many bus stops you should make. If you make too many bus stops, then all that starting and stopping slows you down. That's how we came to six weeks. We need one week to get the plans done, and we need a little more than half a week to stop.

Bill Venners: And that gives you four weeks of execution.

Erich Gamma: Yes, it gives us at least four weeks of execution, which is enough time to do something significant. You have to be able to solve bigger problems in one cycle. We started with four-week cycles, but found the starting and stopping in a four- week cycle was too much overhead.

Bill Venners: Do you have any decompression time built in between these iterations?

Erich Gamma: After we ship a major release, we have around one month of decompression time. We deliver our releases usually in June or July, before people take their vacations. We then have some planned decompression time where we look back and reflect on what worked, what didn't, and what should improve. This is also the time for exploring new stuff. We then transition into thinking about the next release.

Bill Venners: Is there a mini-decompression time at the beginning of each one of the mini-release cycles?

Erich Gamma: You need decompression time when you have really been compressed, but the idea of the milestone releases is to not be continuously compressed. Rather, we want a regular rhythm of sustained development. We start. We stop. At the end before we ship an actual—not milestone—release, then we have some compression. This is the end game, which is intense. We have to do testing and very careful fixing to address last-minute polish or performance items. We know we can only do this for a certain amount of time. The end game is tiring, and afterwards, you need decompression, because everyone has to do a lot of compressed development towards the end.

Bill Venners: What's the end game?

Erich Gamma: The end game is the last six weeks. We have milestone, milestone, milestone, milestone, then basically the last milestone is the end game. It is usually also six weeks. In the end game we want to have convergence. We want to stabilize and move forward carefully. These six weeks are planned in detail and the end game plan is published on the web. The end game plan defines a rhythm of test, fix, test, fix, test, fix. The whole team, supported by the community does that. All the developers and the community test for two to three days, then we fix for four or five days. We test again for two or three days. And, with each fix pass, the rules become tougher for getting changes in. In other words, the later in the end game, the harder it is to get a change in. For example, for a bug to be fixed it needs to be posted to the developer mailing list and the fixes need to be peer-reviewed.

What can also be stressful in the end game is that sometimes we have ripples that have to percolate up the higher layers. We really prefer to work on planned things, but during the end game it is difficult to plan everything.

Bill Venners: What kind of ripples?

Erich Gamma: For example, we find that a lower layer has added a new mechanism that isn't leveraged yet in all the upper layers. These kinds of ripples are difficult to plan. An upper layer might have jumped on a new function and a lower layer thinks they are already done.

Bill Venners: That sounds like a communication issue. The people who are doing the lower layers need to communicate better with the people doing the upper layers.

Erich Gamma: Absolutely, I mention it to show that even after shipping on-time for several years we're not perfect. The best solution we've found is where the lower layers help to carry forward the required changes by providing patches and help to the upper layers. The mindset we're trying to foster is that you are not done with a feature before it has rippled through all the layers.

Agile practices in Eclipse development

Bill Venners: What agile practices does the Eclipse team follow?

Erich Gamma: You'll see our agile practices not only in what we do, but often also codified in the tool we build. We support refactoring, unit testing, fear-free development with a local history and tightly integrated versioning, automated build process support with Ant--things like that--directly in Eclipse.

As far as the agile practices we follow when developing Eclipse, we always test early, often, and automated. For each build we run over 20,000 tests. We have nightly builds that are automated. We get build reports that tell us the failures. Recently in 3.1, we added performance tests. So we not only test for correctness, but also for performance. This has helped us a lot during the 3.1 cycle and actually you will notice significant performance improvements in version 3.1.

We practice incremental design. We invest in APIs up front. But we're also improving them incrementally. We refactor, but when we refactor we keep the stability of already published API in mind. We cannot just break clients. We also eat our own dog food daily. Each component exports its latest plug-ins and runs on top of it. This is great because you are the first one to discover your own errors. And of course when you're building an IDE or tool, you're in a deluxe position to do that. Sometimes we deploy more than once a day. So if we introduced a bug in the last two hours we'll often find them even before the daily build kicks in.

Bill Venners: You actually deploy the components to the developers' work stations, and they are using the currently released version for the ongoing development?

Erich Gamma: Yes, exactly. The developers install the latest built versions of their plug-ins and then run with them.

Customer involvement is an agile practice. And mapped to open source, this means community involvement. And we have a very active and large community. The community isn't shy. It makes ambitious requests, but in the end it is highly rewarding and fun to work with such a community.

Another agile practice is continuous integration. We do nightly, weekly, and milestone builds. We also have a notion of build promotion. Nightly builds build the latest code. We use it as an early warning system to uncover integration problems across components. The next promotion level is an integration build: once a week, we want to have something that is good enough for other eclipse committers to use. The next promotion level is the milestone build, which is every six weeks. Here we want to have a build that is not just good enough for us committers, but is good enough for the entire community. So we practice continuous integration with promotion.

We also practice short development cycles. Our development cycles are not as short as XP suggests. As I explained with the bus stop metaphor, we feel we can't go shorter than six-week cycles.

We also do incremental planning with time boxing. Given that we have the six-week cycles, this is our time box. Our top-level plan is also a dynamic plan that is incrementally updated. That is, we don't come up with a final plan up front. We have an initial plan that lists the milestones, the themes and major work items, but we adapt the plan over time. This was also a lesson learned. Originally we had static plans, which meant the plan was accurate when we started, but not later on. We opted to plan as we go, and now the final version of the plan is done the day we ship. All of our planning is done transparently. For each plan item we state whether the item is committed or proposed and we update the status regularly. Therefore the community sees what we have committed to, and what we have proposed. We usually want to hit our target, and if we can't hit it, we are willing to cancel proposed items, but we really hate to drop committed items. Looking back on 3.1 I'm glad to see that this was indeed rare.

Next week

Come back Monday, July 11th for the next installment of this conversation with Erich Gamma. If you'd like to receive a brief weekly email announcing new articles at Artima Developer, please subscribe to the Artima Newsletter.

Resources

[1] Erich Gamma is co-author of Design Patterns: Elements of Reusable Object-Oriented Software, which is available on Amazon.com at:
http://www.amazon.com/exec/obidos/ASIN/0201633612/

[2] Erich Gamma is co-creator of JUnit, the defacto standard Java unit testing tool:
http://www.junit.org/index.htm

[3] Erich Gamma leads the Java development effort for the Eclipse tool platform:
http://www.eclipse.org/

Talk back!

Have an opinion? Readers have already posted 3 comments about this article. Why not add yours?

About the author

Bill Venners is president of Artima Software, Inc. and editor-in-chief of Artima Developer. He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project, whose ServiceUI API became the de facto standard way to associate user interfaces to Jini services. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community.