This article is sponsored by the Java Community Process.

A Conversation with Terence Parr

---

Since Trygve Reenskaug first described the separate concerns of model, view, and controller in 1978, keeping model code independent of view has become widely adopted developer wisdom. Yet, popular enterprise frameworks provide plenty of ways in which a developer, intentionally or by accident, can mix view and model or controller code.

With a background in language theory and compilers, and a need to build a maintainable Web site infrastructure for a business, Terence Parr designed StringTemplate, a templating system that strongly encourages a complete separation of model and view code. An open-source project, StringTemplate is used not only in Web applications, but as a template generator of the popular ANTLR Java parser generator, another one of Parr's creations.

Parr is an assistant professor of computer science and director of the graduate program at the University of San Franscisco's Computer Science department. He was also founder and chief architect of the popular Java developer site, JGuru. Partly as a result of Parr's academic background, StringTemplate is built on solid computer science theory. At the same time, it is also a very practical tool for building enterprise Web applications or for code generation.

In the first part of his interview with Artima, Parr discusses the importance of model/view separation, and points to ways of enforcing such separation even when many programmers work on an application.

Separating model and view

Bill Venners: In Enforcing Strict Model-View Separation in Template Engines, you wrote, "The mantra of every experienced web application developer is the same: thou shalt separate business logic from display." Why?

Terence Parr: There are a number of problems. There is a practical problem in a large company with a large website. Who is doing the HTML? Hopefully it is not the coders, because we suck at that. That is not our job. We need a designer to do that. The more your templates look like a program, the less the designer will be able to actually work in parallel with you.

My experience is that designers don't understand loops or any kind of state. They do understand templates with holes in them. Everybody understands mail merge. And if you say, "Apply the bold template to this hole," they kind of get that, too. So separating model and view addresses this very important practical problem of how to have designers work with coders.

The other problem is there is no way to do multiple site skins properly if you don't have proper separation of concerns. If you are doing code generation or sites with different skins on them, there is no way to properly make a new skin by simply copying and pasting the old skin and changing it. If you have the view and the logic together, when you make a copy of the view you copy the logic as well. That breaks one of our primary rules as developers: have only one place to change anything.

Bill Venners: What belongs in the M, V, and C of MVC?

Terence Parr: Model-View-Controller was dreamed up in the '80s during the Smalltalk days for producing GUIs. They had a model, such as a table of data. The view was the widget that displayed the data in the model. The controller would listen to events and route them. Given an event, the controller might tell the model data to update, which might cause another event telling the view to update itself. So MVC was like a little triangle that would constantly feed events around. The controller said how to hook what data to which view.

For the "MVC" of a web app, I make a direct analogy with the Smalltalk notion of MVC. The model is any of the logic or the database or any of the data itself. The view is simply how you lay the data out, how it is displayed. If you want a subset of some data, for example, my opinion is that is a responsibility of the model. The model knows how to make a subset. You should not be asking your graphics designer to filter a list according to age or some other criteria.

The controller in a web app is a bit more complicated, because it has two parts. The first part is the web server (such as a servlet container) that maps incoming HTTP URL requests to a particular handler for that request. The second part is those handlers themselves, which are in fact often called "controllers." So the C in a web app MVC includes both the web server "overlord" that routes requests to handlers and the logic of those handlers themselves, which pull the data from the database and push it into the template. This controller also receives HTTP POST requests and processes these, sometimes updating the database.

I look at a website as nothing but a graph with edges with POSTs and GETs, and that routes pages.

Bill Venners: Can you elaborate on the graph? How does that look?

Terence Parr: That is a pretty neat thing. Let's say you have a page that has links labeled "View Inbox" and "Go to My Profile," and a button labeled "Submit." In the graph, there would be two GETs emanating from that page, and one POST, each of which take you to another page.

Or, imagine you want to buy an airline ticket. There is the entry page where you start, and then another page and another and another. On any of those pages you could cancel to go back. Each one of those forward or backward motions ends up being a transition in the graph for that site.

It could also be cyclic. You might be sent back to the same page to repeatedly edit a forum post. Each one of those operations is a transition in the graph. You come out with a giant graph that describes all possible motions on a website. Each page of a website is a node, and each operation or transition between the pages is a line between nodes. It is a really nice description of a website, a real site map.

Separation cost and benefit

Bill Venners: You wrote, "Programmers often view strict separation as costing more time and hassle. While this may be true in the small, such as adding a new piece of data to a view, my experience is that projects progress much faster in the long run, and result in much more flexible, robust code." Can you talk about the tradeoffs here?

Terence Parr: I view it as the agile programming concept. I believe the early XP guys pointed out that there is a paradox. Imagine you have infinite time to write software. You'd talk to the customer a lot. You'd write documentation. You'd do all these things that seem locally non-optimal, but the paradox is that globally it's optimal and you get done faster with more robust and better code.

We have all been involved in a project—maybe you wrote it or maybe you inherited it—but the code is just like spaghetti. Working on such code is like walking through foot-deep snow as opposed to hard-crusted snow. It's a small difference. In both cases you're walking on snow, but one is crusted and one isn't. You can make a lot more progress on the crusted snow. If you take the long view, if you plan for the future, in general you are going to be better off. The same is true about strictly separating model and view.

If you just want to grab some more data from the database and stick it in this view, it is fast and it's locally optimal to do that right from the view. You can beat me every time doing that. Well, not every time, but most of the time you are going to win, because it takes longer to grab it in the controller and inject it into the view as a parameter, especially if a designer has to go ask a programmer for it.

In the long view, having strict separation of concerns with no logic or data computations in the view, buys you so much. The graphics designer can actually look at the design. Strict separation ultimately produces a better product. The code is more robust and it is easier to change. It in the long term you are actually faster.

Bill Venners: You wrote, "While programmers value simplicity, they more often value powerful functionality and amazing 'one liners,' incurring the cost of complexity." How so?

Terence Parr: Take academic journal papers, for example. I am continually amazed when I read an academic paper and they say, "Look how much I can do on this one line!" and I say, "Yes, but, see, nobody does that." Programmers write 100,000 line programs. The key is how flexible is it and how well does it fit into their environment. Is that one liner written in ML? That is fine, but it doesn't fit into our world.

People fear being painted into a corner. Programmers are naturally afraid of lack of power. I think we all pay lip service that it should all be simple. But the first thing a programmer is going to say to me is, "Wait a minute, what if I can't do this?" I might respond, "Well, I'm trying to hit 95% with insane simplicity."

I love the fact that some guy on the Velocity list said, "The good thing about Velocity is that the grammar can fit onto the back of a matchbook." I said, "Really? You're talking to a grammar guy here and I know you're wrong because I'm looking at it! You just add for loops and 'Boom!' you blew the grammar past matchbook size."

I only have four operations in string template. Yes, it is a little weird and you have to wrap your head around the simplicity—yet it is so powerful. I think it is a better way to go.

The slippery slope

Bill Venners: You wrote, "To support harmless functionality such as 'make all negative numbers red,' engines support attribute value testing, which leads to egregious violations such as 'if user is James and host is wraith.'" I hear the slippery slope argument in there, which is usually considered a logical fallacy. Someone might say, 'Oh, no, we can't legalize marijuana because then all drugs will be legal in a year.' And the question is, 'Well, will they?' Why do you think the slope is slippery for template engines?

Terence Parr: That is a different kind of a question. You would have to legalize four or five drugs, whereas in the template engine case, it is binary. You only have one choice. Do you allow integer comparisons?

Bill Venners: Yes, but if you allow integer comparisons, what you are saying is that before long programmers will start writing scary code in there.

Terence Parr: A lot of people are all about 'the power,' right? I see it every day. In the original version of StringTemplate, I gave people the ability to test the absence or presence of an attribute, and that was it. So if a boolean attribute was false, but existed, the if expression came out true because the boolean attribute was present.

People said, "That's crazy. You've got to be able to test the value of a boolean." I said, "You know, you are absolutely right." It was just too weird and non-intuitive to do things that way. So I fixed that and then people said, "Well, why can't I just do this? It's just this." I get it every day on the mailing list. It's like a little kid pushing you every single day. "Dad, can I have this? Can I have this? Can I have this?" It actually really does exist.

As a tool developer, whether it is ANTLR or StringTemplate, I get nonstop requests for more stuff just because they want it in that situation. If we isolate it to just one binary choice: Do you allow people to check for negative numbers and make them red? It seems innocuous. And it is. There is nothing wrong with that. However, it also allows me to check for age and decide that if your age is over 100 we are not going to insure you anymore. Then you have this constant 100 sitting in the template and that's business logic. It really has to be in the model.

Bill Venners: I bought a software product with JSPs that contained business and display logic very intermixed. Maybe they were written before it was common knowledge this was a bad thing in web apps. I tried to separate it over time, but it was just ugly. Then we switched to Velocity because [Velocity creator] Geir Magnussen actually sat down with me. We had coffee, and talked about how awful JSP is because it mixes code and data, and he convinced me.

Terence Parr: All those Velocity guys and I agree in the principles...

Bill Venners: So I switched, but then I found myself...

Terence Parr: Taking the expedient route.

Bill Venners: I am not sure. For example, I called a lot of static methods to do things like converting a camel case string to constant form. So we had all these static methods, and you had to compile them first. We did this perhaps in part because we didn't realize we were mixing business and display logic, and perhaps in part when we were in a hurry.

Terence Parr: It is expedient.

Bill Venners: So it comes down to what you think is realistic. If programmers do believe that view and model should be separated, why can't they do it with JSP or Velocity? Why can't they discipline themselves?

Terence Parr: Oh, they could, but it never happens. That is the problem.

Bill Venners: Why?

Terence Parr: It is some psychological thing. Everybody knows we shouldn't speed: it's bad for gas mileage, it's dangerous. But not everyone agrees that speeding is always a bad thing.

Bill Venners: Yes, one problem is probably that not everyone agrees that you should separate business and display logic.

Terence Parr: Yeah, not everyone agrees you should drive the speed limit either. I agree, and most of the time I drive only a few miles over the speed limit. But sometimes I am in a hurry. I won't tell about some of the times I have been caught going faster than the speed limit. I've never had a ticket, but the fact is that when I am in a hurry, those rules be damned. And I could have paid the consequences, with either a ticket, by crashing my car, or by hurting somebody, including myself.

I'm a very disciplined programmer, but I am also susceptible to expedience. It is just the way we are. Think about the average programmer. They don't even understand the principle. They are just looking for a way to say what they need to say and they don't think about architecture. So I am trying to actually force good behavior.

There is huge difference between encouragement, as Velocity does—despite their manual saying that they enforce—and enforcement. The difference between encouragement and enforcement is vast. It is like being one penny in debt and one penny above—one is a negative derivative and the other has a positive slope. There is a huge difference. I have had to make realistic gray-zone decisions. What can I allow to make programmers' jobs easier that does not open a huge gaping hole that allows abuse? I have added features to make using StringTemplate more reasonable.

Designing the easy route

Bill Venners: When you hire a bunch of programmers it can be quite difficult to get them all to go in the same direction. One of the ways I think about architecture is that you can try to define it such that it is easy for people to go the way you want them to go, and painful, though likely possible, to go other ways. People will usually follow the easy route.

Terence Parr: That's right.

Bill Venners: For example, in StringTemplate I could override toString() if I wanted to.

Terence Parr: To erase the hard drive from a template.

Bill Venners: Right, but it is kind of painful.

Terence Parr: Exactly, you might as well do it the right way. That is basically what I decided to do with those slippery slope features. I've tried to simply make it painful for you to do the bad thing, and nice to do the good thing.

Bill Venners: One of the programmers who worked on our code commented about StringTemplate that he would rather discipline himself than have the tool discipline him. Another one said he agrees with the strict separation of model view, but he thinks you need to have some logic in view to make it easier to do stuff.

Terence Parr: Well, there is some logic.

Bill Venners: You said that a model-view duo achieves essentially unrestricted power without violating rules of separation. To what extent are you limited by this strict separation?

Terence Parr: I believe in principle there is no possible restriction, because I can compute whatever I want in the model and push it into the template. However, remember I can't actually compute any HTML in the model and push it in, because that would violate the separation of model and view. So it is not completely obvious that there is no restriction, and I hesitate to dismiss that question with that simple argument of "Well, you can compute whatever you want in the model and push it in." I didn't want to open that can of worms in the Enforcement paper though [see Resources], because I already had to do one proof. I can give some anecdotal evidence instead.

The back end of ANTLR is all written in StringTemplate, and all of the back ends, with a few minor exceptions—a couple of methods to translate character definitions or whatever, are all templates. People who make back ends to target other languages, like C++ or Python, follow a template interface I provide. They need not modify any code at all to target another language.

Believe me, these are extremely complicated templates. There are hundreds of templates. Generating code for a parser generator is extremely complicated, but in ANTLR 3 it is done with one model and controller and multiple views. So it actually works. When I change logic in the code generator, everybody else just changes automatically. They don't have to worry about it.

The other thing is, if you want to be disciplined yourself, then you shouldn't mind me enforcing it, if your desire is really true. On the other hand, you probably want to bend the rules sometimes. That is probably this gentleman's argument. Sometimes you do want to bend the rules, and it is reasonable.

Bill Venners: Sometimes, right before release, you do the expedient thing, and that is good for the business because you meet the deadline. Later you can spend some time later cleaning it up...

Terence Parr: But do you? The business says let's move on to the next thing.

Bill Venners: Strict enforcement doesn't let you do that. You have to take the time to do it right the first time.

Terence Parr: Again, the paradox is that, long term, you don't have to go back and refactor that.

Resources

StringTemplate

http://www.stringtemplate.org

Trygve M. H. Reenskaug's home page
http://folk.uio.no/trygver/

Terence Parr, "Enforcing Strict Model-View Separation in Template Engines"
http://www.cs.usfca.edu/~parrt/papers/mvc.templates.pdf

Talk back!

Have an opinion? Readers have already posted 13 comments about this article. Why not add yours?

About the author

Bill Venners is president of Artima, Inc. He is author of the book, Inside the Java Virtual Machine, a programmer-oriented survey of the Java platform's architecture and internals. His popular columns in JavaWorld magazine covered Java internals, object-oriented design, and Jini. Bill has been active in the Jini Community since its inception. He led the Jini Community's ServiceUI project, whose ServiceUI API became the de facto standard way to associate user interfaces to Jini services. Bill also serves as an elected member of the Jini Community's initial Technical Oversight Committee (TOC), and in this role helped to define the governance process for the community.