"What is the best UML tool?"
"A white board marker."

Introduction

Guerrilla Development is software development in an environment unsupportive of the effort. In my first few essays I discussed some of the problems software departments might have in their interactions with the rest of the company. These included stonewalling product managers, poor-planning upper managers, and even hostile engineers in related departments or disciplines. Software engineering discussions in books, magazines, and user groups tend to apportion too much blame on the software department when things don't go smoothly. I believe that there is plenty of blame to go around.

That said, I agree that some of our problems are self-inflicted. Other problems might be better categorized as missed opportunities. In this essay, I'll discuss one such missed opportunity and articulate some of my worries for the future of UML.

The Future of UML

The reactions that I get from many engineers when I broach the topic of UML leads me to believe that they think the whole thing is a conspiracy foisted upon the software community by the tool vendors in their endless search for more suckers. If that notion prevails, I'm certain that we will have lost out on something important.

UML can be a fabulous tool for helping engineers communicate ideas. The power of UML isn't the fact that it is drawn with boxes and arrows but that the specific shapes of things and the types of arrows have some agreed upon meaning. If two engineers know something about, say, class diagrams in UML and one of them begins drawing a class hierarchy, the other immediately knows which class inherits from the other. There is nothing magic about the boxes or arrows; it is just a set of agreed upon conventions that allows engineers who share the knowledge of those conventions to communicate quickly and effectively.

As simple as all this sounds, the influence of the tools vendors, book publishers, and consulting houses trying to make UML sound elaborate, complicated, and expensive cannot be ignored. Their efforts to take these simple conventions for communicating engineering design ideas and turn them into something profitable may endanger the entire initiative. Is UML worth saving? Can it really help real projects, projects that produce code, have deadlines, and live in Guerrilla Development environments?

Yes. I believe UML can help. And, as always, I've got a few stories from my lurid and sordid past to back up that claim.

The Prescriber

I began working for a company that made virtual private network software in the Spring of 1998 and was teamed up with another fellow, Mike (also a new hire) to work on a part of the project called the Prescriber. The Prescriber was an interesting concept. It was to be part of the VPN dialer residing on the client's MS Windows machine that would diagnose and automatically repair (if possible) problems with the client machine's configuration, software, or settings. The idea was this: some of the problems a client might have with his laptop on the road were minor such as well-known .DLL conflicts or improper network settings. A small, extensible rules engine might be able to detect and even correct many of these problems.

The initial design of the Prescriber had been done in TCL (as one of the other fellows who worked at this company had been a college classmate of John K. Ousterhout, the developer of TCL). Mike and I wanted to do the next phase in Java. After some discussion, the decision was eventually made to transition the project to Java and work began in earnest.

Mike and I quickly found that we worked well together. We began camping out in one of the smaller conference rooms and filling the white boards with ideas. One of the unique aspects of this part of the project is that the results had to be user extendable. For example, customers might wish to use the Prescriber to help enforce their IT policies. The Prescriber wasn't going to just help establish the VPN connection for the client; it was also going to help manage the entire VPN experience. This extendibility requirement placed some extra burdens on us that other members of the development team did not have.

For one thing, our program documentation would need to be much more extensive than anybody else's. This part of the project was not just a binary-and-nobody-will-see-the-insides-of-this deliverable. We had to have a clear structure for the facility with an architecture that would enable, and even encourage, extendibility. We also had to plan on having a significantly more detailed documentation set including a Theory of Operation document so developers in those IT departments would have some idea of how to create these Prescriber extensions safely. Creating user-extendable software is not a trivial task. Nardi's book A Small Matter of Programming discusses this at length.

Mike and I needed to work together efficiently and, hopefully, create artifacts during our design and implementation phases that could be reused later when we created the final documentation. Most of all, we needed to run this project so we could get some version of the Prescriber done for the next release of the dialer, an event only months away.

It is the Code, Stupid

We needed code and we needed back-up documentation. We also needed a way to divide up the project so that Mike and I could work quickly without "stepping on each other". I began drawing UML class and package diagrams on the white boards during our marathon design sessions. I'm not sure how much UML Mike had seen before this point but he was extremely quick to catch on. We did refinement after refinement on those white boards, seeing the structure of the product and the relationships between the pieces. We found a convenient way to divide the problem down the middle.

Things were looking up but, of course, we had yet to write a single stitch of code. We needed a way to be as productive in the detailed design and coding as we had been in this early architectural and high-level design phase. The UML diagrams first drawn on the white boards in the conference rooms had been transferred to my notebooks but it seemed like a waste to stop there. The descriptions and relationships in those diagrams had been instrumental in helping Mike and I understand the problem and our solution. This is exactly the kind of insight that an IT person extending the Prescriber later would need. Why can't we carry this forward?

I had some experience with UML tools from previous projects. Most were bulky, buggy, and slow (not to mention expensive). There was one that had caught my eye, however. The tool's name was ClassDesigner and it handled only a couple of the UML diagram types (code and package) but it had rock-solid round-trip engineering facilities. That is, you create your design using class diagrams, use arrows and lines to indicate the relationships between the classes (inheritance, dependencies, references, etc.) and then push the button to have the tool generate the corresponding Java code. After you've made changes to the code (filling in the implementations for the methods between the curly braces) you can update the diagrams and push the button again and the code is updated. This means your drawing and the associated code can stay synchronized.

This round-trip engineering (being able to continue to generate the code from the drawings) meant that a couple of our biggest problems would be solved:

• Mike and I were making changes constantly. This was a way for us to keep each other informed and to document the changes.
• There wasn't going to be a lot of time after the coding was done to produce the Theory of Operations document. We'd need to have as much of the materials in place as possible. Having the diagrams "track" the code meant that they could be used later with high confidence.

I was convinced that ClassDesigner would do the job for us. But, I had to convince Mike (who was pretty easy-going and thought it was worth a try) and management. Given that I had spent a lot of points getting the Java (and not TCL) decision, I wasn't sure if I could go to the well again so soon. I did. And, to my pleasant surprise, the VP of Engineering was intrigued by the idea. Of course, he wanted to see more. I agreed to give him a demonstration of what this tool could do.

The other feature a typical UML tool has is reverse engineering. The tool reads all of your source files and creates appropriate class diagrams. I reverse engineered the entire project.

As with most projects, some aspects of the development were going better than others. There was one particular part of the project (no, I'm not going to say which one!) that had been a point of contention. As I untangled the diagrams for the various subsystems, some areas of the diagram became neat and pretty. Other areas were a bit jumbled but still intelligible. One area was a mess. Guess which part of the project that was!

Early on a Saturday morning I set up the projector and prepared for my 1:1 with the VP. When he arrived, I put up a single drawing that showed a 100,000 foot view of the project. You could see boxes and lines but no names. Most of it looked organized but one area looked completely tangled. He saw it immediately.

"My theory", I told him, "is that the design isn't right if the picture isn't pretty. If the elements of the design have too many interdependencies or if there are too many moving parts that will show up in the diagrams."

"What's this area here?", he asked. It was the allegedly problematic part of the project. "I wish we had this tool in the beginning", he said. I knew I'd sold him on the idea. He agreed to get a small number of licenses for the tool.

We produce products that are made of code. But, the structure of these products, the high-level design and architecture, is not easily viewed by just looking at the code. This 100,000 foot view, or topsight as Gelernter describes in his book Mirror Worlds is more easily obtained through something like UML. The VP had gained topsight into his project, possibly for the first time, and he recognized its value immediately.

Resistance

My story has been entirely too happy so far. Except for some skepticism from the VP, we had not run into any resistance concerning UML or ClassDesigner. Once we got the go-ahead, however, there was the beginnings of grumblings about the newfangled tool. Luckily, our part of the project was isolated enough that few felt threatened by it. The most resistance came from the vendor.

Cayenne Software had purchased the ClassDesigner product from a one-man company run by a fellow named Naresh Bhatia who followed his product to Cayenne. Bhatia may have understood what he created but it was clear that few others at Cayenne did. When I talked to Cayenne's sales team about obtaining licenses for the ClassDesigner product they were confused. Was I sure? Didn't I want the more "capable" (expensive) ObjectTeam product? "No", I insisted, ClassDesigner does exactly what I need. It is fast, reliable, easy to learn and use, and is a quarter the price of ObjectTeam.

To be honest, I don't remember how this happened but somehow, I got roped into doing a full-blown product comparison between ClassDesigner and ObjectTeam. Perhaps I had mentioned to the VP that the vendor was pushing this or, more likely, the VP was in on one of the sales calls and was intrigued by the ObjectTeam pitch made by the Cayenne sales person.

I spent a week carefully comparing the products. I took the UML specification and found that ObjectTeam couldn't actually produce most of the drawings in the specification. The round-trip engineering was weak (which was the key feature we were looking for), the interface was muddled, and it had a nasty tendency to crash just before I had thought about saving my work. I produced a 30 page report highlighting the deficiencies of ObjectTeam. I heard later that my report had circulated through some of the highest circles at Cayenne. The defects and deficiencies were undeniable; they had just been conveniently "omitted" from any previous status reports from the ObjectTeam group, apparently.

With this behind us, ClassDesigner was officially selected as our UML tool and Mike and I proceeded to complete the project. In fact, it worked so well that the VP bought into my suggestion that we obtain a site license for the product and begin encouraging other engineers to use it. I negotiated the deal near the end of Cayenne's next quarter.

Then Cayenne was bought by Sterling Software. Sterling, confused why anyone would buy ClassDesigner when they could pay four times as much and get ObjectTeam, promptly killed the product. Furious, I called my Cayenne (now Sterling) sales person and demanded that we be refunded our site license fee since the product had actually been killed before we had a chance to deploy it. Our money was returned.

Still hoping to keep UML and UML tools in the loop, we briefly played with Rational Rose which, like ObjectTeam, was large, slow, expensive, and buggy. But, the bloom was off the rose (so to speak) and the efforts to integrate UML into the development process were abandoned.

Observations and Lessons Learned

There are several things you might take away from this story:

• White board markers make powerful UML tools One of the most important phases of the project was the very beginning when Mike and I needed to get organized, create a workable architecture, and divide the project up so we could work quickly and efficiently. The UML tool we used was a white board and a set of white board markers. I contend these tools are still the best tools available today.
• Most have never seen effective use of UML My manager was delighted to see a reverse engineered drawing of his project. For the first time since its early days, he believed he could see what was going on. Properly done, UML can provide what Gelernter calls topsite even late in a project's development cycle.
• UML tools can be used day-to-day in start-ups Sometimes I think the UML vendors believe that only Fortune 500 companies and government contractors buy these tools. Well, maybe that's right. But, I have existence proofs that projects in high-stress start-ups can also be helped greatly by the right tools. ClassDesigner helped us kick butt, a fact that was apparently mystifying to its vendor.
• Not all engineers are good designers Yeah, there's a shock! Without a reasonable format for articulating the details of a design, however, it might be hard to tell the difference between a weak design and a weakly presented design. UML can help provide a common structure and tool set for designers to exchange ideas. I'm convinced that most weak designs are allowed to proceed to the next phase because the reviewers never really understood what they were approving. If UML can provide clarity during reviews, it might be well worth the effort.
• Vendors are idiots A bit harsh, but too near the truth to dismiss outright. If anybody on the Cayenne sales team had done the simple things I had done (like just try to draw the sample diagrams from the UML specification with ObjectTeam) they would have easily come to the same conclusions I did. But, nobody tried. The sales people had no idea whatsoever how this product was going to be used day-to-day by us. And, when we explained it, they didn't believe us.
• UML's future depends on breaking out of this tool idiocy UML can be a powerful means of expressing design ideas but its future is bleak if the kind of idiocy described here is left unchecked. The tool vendors need to understand that developers respect tools that help them crank out code. Any tools developed need to be rock solid, fast, affordable, and adoption-friendly. From what I see, we've got a long way to go.

Postscript

I have a very good friend (Jeff) who also happens to be an excellent engineer (and engineering manager). One of the things that makes Jeff an interesting person to talk to is the fact that he sees the world much differently that I do. He is a very (very, very) verbal person. He gains nothing from diagrams; he'd rather read the text. Pictures convey very little information for him.

I have a season ticket package with the Boston Symphony Orchestra and asked Jeff if he and his Significant Other wanted to join me and mine to a concert. "No", he replied. If the music doesn't have words then, to him, it is just "a lot of pretty sounds". It is nice, of course, but it doesn't hold any meaning to him. I'd never thought about how visual music could be until that point, but understood immediately why he might be reluctant to go. Jeff is the kind of thinker that prefers literature over music because that is how his brain works.

Trying to do a white board session drawing UML diagrams with Jeff would probably be an exercise in frustration. I'm sure there are plenty of engineers out there who, like Jeff, are much more verbal and far less visual than I am. For those folks, I conceded that UML has very little to offer you. This is one (of many) reasons why UML cannot be a substitute for written artifacts.

Final note

I ran into somebody who had still been associated with the project this summer, some 4+ years after the first new Prescriber had shipped. "Had the documentation been helpful?", I asked. "Are you kidding?!", my friend replied. "It scared the hell out of 'em." I looked aghast. He continued, "they take one look at the size of the notebook you left and they say I'm not touching that! It looks complicated!." Sigh...