Artima.com has published Part IV of an interview in which Martin Fowler discusses design decay, the cost of flexibility and reusability, four criteria for a simple system, and test-first design of interfaces.---xj40dkcfea73---Artima.com has published Part IV of an interview in which Martin Fowler discusses design decay, the cost of flexibility and reusability, four criteria for a simple system, and test-first design of interfaces.

http://www.artima.com/intv/flexplex.html

The cost of flexibility is complexity. Every time you put extra stuff into your code to make it more flexible, you are usually adding more complexity. If your guess about the flexibility needs of your software is correct, then you are ahead of the game. You've gained. But if you get it wrong, you've only added complexity that makes it more difficult to change your software. You're obviously not getting the payback.

It's not hard to guess wrong about flexibility needs. You can guess wrong if requirements change. What you think is a requirement for flexibility now may go away or change in the future. You can also guess wrong if you put extra code into the program to improve flexibility but you don't get it quite right. You get more complexity without getting the flexibility you were after.

The alternative is to use the XP approach and not put the flexibility in at all. XP says, since most of the time we get it wrong, just don't put the flexibility in there. Now if you can't evolve your design safely, then that is a foolish route to take. But if you can evolve your design safely, it becomes quite a nice approach. In fact it becomes a self-reinforcing approach. If you strive to keep your design as simple as possible by avoiding speculative flexibility, then it's easier to change the code because you have less complication to deal with. The code is easier to understand and easier to change. As a result, you can make changes much more quickly.

What do you think of Martin Fowler's comments?

I've been reading a lot about XP lately. I'd like to know if the XP beliefs surrounding design apply to a database as well. For example, say I'm designing some app that involves a decent-sized database. Should the same principles apply to my database design as apply to my code design? Changing code later is one thing, but changing the database may be a different story.

Maybe you could ask Martin Fowler this for the next interview in the series -- if refactoring applies to designing a database as well.

Interesting comments from Martin Fowler, as usual. His point that the Gang of Four book is basically about the use of interfaces is very important. There's now at least one other book that emphasises the value of interfaces: my new book on J2EE, "Expert One-on-One J2EE Design and Development" has a lot to say on the merits of interfaces. See http://www.wrox.com/books/1861007841.htm. In particular, it looks at how JavaBeans can be used to maximise the benefits of interfaces.

I think the main reason that many developers under-utilize interfaces is that they perceive it increases complexity. Through a consistent infrastructure framework such as I discuss in the book, and the use of JavaBeans, the complexity can be avoided.

actually there already have some discussions about this topic, but probably not easy to put in practice

http://www.agiledata.org/essays/databaseRefactoring.html

> I've been reading a lot about XP lately. I'd like to know
> if the XP beliefs surrounding design apply to a database
> as well. For example, say I'm designing some app that
> involves a decent-sized database. Should the same
> principles apply to my database design as apply to my code
> design? Changing code later is one thing, but changing
> the database may be a different story.


The short answer is that you can build up a database design in an evolutionary way as well. My colleague Pramod Sadalage is starting to write up how you can do this based on his experiences on a sizable project. He's started a discussion group at http://groups.yahoo.com/group/agileDatabases . I'm hoping to get an article on the subject on my web site soon

The discussion of "Flexibility and Complexity" and "Flexible versus Reusable" is a very good explanation of how to have flexibility code with simple design, which XP suggesting. But it is hard for me to practice, since I touch computer programming, what I learn is put more time to design, think about the flexibility and reusability.

As a long time practitioner of the "quality before design" discipline, I have a serious problem with XP's insistence on 'emerging' design. The differences between XP and the quality-before-design (QBD for short) schools are fundamental and irreconcillable. I've reached this conclusion after several months of intense debate with several keen practitioners of XP.

At this time, I'd like to know what are XP's "high priesthood", such as Fowler et al, doing to address the objections posed to them by the QBD camp?

To refresh your memory, what we (the QBD-ers) are big on is the fact that, rather than view our profession as being akin to caterers, that is, catering to our customers' whims, we prefer to view ourselves as being akin to nutrition specialists. That means that we're telling our customers something along these lines: "Of course we realize that you know what is it that *you like*. However, we think that you may not always know what is it that's *good for you*. We are the experts that specialize in knowing what's good for you."

The above is similar to how nutritionists in general operate -- they do not observe what is it that their patients like to eat. Instead, they prescribe the diets that are the healthiest choice for the given patient. In the process, they totally ignore any objections that may occur.

In my conversation with XP proponents, they all seemed to assume the stance of the caterers -- we'll deliver whatever it is that the customer desires. They seem devoted to being extremely sensitive to their customers' wishes. I think that's utterly wrong, because the customers in most cases have develped notoriously bad, if not right down dangerous habits, and catering to those habits equates to severely damaging their wellbeing.

We, the information services specialists, must take the initiative to ensure that our customers get the highest quality possible. In order to achieve that, we must focus solely on the quality (whatever 'quality' may mean in a given situation). Only once the quality has been established, do we allow the process to move on to the design (that is, elaboration) and construction phase. At which point we let other sets of criteria govern the remainder of the process (as it gets largely irrelevant to the quality).

In other words, we don't fret so much over the quality of the code per se. We worry about the quality of the intentions as implemented by the code. And we don't believe that that quality could be governed by the code itself, nor by the manner in which that code was produced.

It's similar to delivering a new car to the customer -- what's important is that the car satisfies the criteria for highest possible quality. What methodology was used when building that car is of secondary importance. The quality of that car was unambigously established long before anyone put any design efforts on paper. Or else the car would end up being dysfunctional.

Alex

As a long time practitioner of the "quality before design" discipline, I have a serious problem with XP's insistence on 'emerging' design. The differences between XP and the quality-before-design (QBD for short) schools are fundamental and irreconcillable. I've reached this conclusion after several months of intense debate with several keen practitioners of XP.

At this time, I'd like to know what are XP's "high priesthood", such as Fowler et al, doing to address the objections posed to them by the QBD camp?

To refresh your memory, what we (the QBD-ers) are big on is the fact that, rather than view our profession as being akin to caterers, that is, catering to our customers' whims, we prefer to view ourselves as being akin to nutrition specialists. That means that we're telling our customers something along these lines: "Of course we realize that you know what is it that *you like*. However, we think that you may not always know what is it that's *good for you*. We are the experts that specialize in knowing what's good for you."

The above is similar to how nutritionists in general operate -- they do not observe what is it that their patients like to eat. Instead, they prescribe the diets that are the healthiest choice for the given patient. In the process, they totally ignore any objections that may occur.

In my conversation with XP proponents, they all seemed to assume the stance of the caterers -- we'll deliver whatever it is that the customer desires. They seem devoted to being extremely sensitive to their customers' wishes. I think that's utterly wrong, because the customers in most cases have develped notoriously bad, if not right down dangerous habits, and catering to those habits equates to severely damaging their wellbeing.

We, the information services specialists, must take the initiative to ensure that our customers get the highest quality possible. In order to achieve that, we must focus solely on the quality (whatever 'quality' may mean in a given situation). Only once the quality has been established, do we allow the process to move on to the design (that is, elaboration) and construction phase. At which point we let other sets of criteria govern the remainder of the process (as it gets largely irrelevant to the quality).

In other words, we don't fret so much over the quality of the code per se. We worry about the quality of the intentions as implemented by the code. And we don't believe that that quality could be governed by the code itself, nor by the manner in which that code was produced.

It's similar to delivering a new car to the customer -- what's important is that the car satisfies the criteria for highest possible quality. What methodology was used when building that car is of secondary importance. The quality of that car was unambigously established long before anyone put any design efforts on paper. Or else the car would end up being dysfunctional.

Alex

Alex,

I believe - using your analogy - your assumption that we are nutritionalists are wrong. We ARE caterers!
If we were trying to solve technology problems alone, you would be right. We do know more about technology than our customers.
BUT we are trying to solve business problems. And we don't know the customer's business - he does! So we listen to him and use our technology expertise to decide how to give him exactly what he wants.

> To refresh your memory, what we (the QBD-ers) are big on
> is the fact that, rather than view our profession as being
> akin to caterers, that is, catering to our customers'
> whims, we prefer to view ourselves as being akin to
> nutrition specialists. That means that we're telling our
> customers something along these lines: "Of course we
> realize that you know what is it that *you like*. However,
> we think that you may not always know what is it that's
> *good for you*. We are the experts that specialize in
> knowing what's good for you."
>
This paragraph reminded me of Ken Arnold's discussion of the arrogance of taste, which is on this page:

http://www.artima.com/intv/taste.html

I have found the process of getting requirements out of customers has mostly to do with human relationships and little to do with technology. In my experience clients don't usually know exactly what they want, but they usually have general ideas about what they want. Also, different people usually have different pet features they understand in more detail that they want included in the requirements. Often the client doesn't know as much about technology as I do, but they know far more about their business than I do. I see my role as a technical consultant as helping them figure out how best to achieve their business goals given the current technology.

What I find helpful is bringing all the parties together who have an opinion and want to feel included in defining the requirements, and get everybody talking about the requirements. I explore the requirements space together and help them figure out what they want. It is a process of exploration and negotiation between all the parties in the meeting. I do make suggestions and propose ideas, but for the most part the process feels like listening. After such a meeting I'll write down what I think we all agreed to and send it to everybody who attended the meeting.

My intuition is that it might make sense for developers to see themselves as nutritionists in the area of technology, but not in the area of the client's or employer's business.

As Bill noted, the requirements gathering and high-level design process needs to be a conversation between the customer and the development team.

But in the end, the customer is king when it comes to externally observable system behaviors. For all other behaviors, and for implementation costs, the development team knows best.

Saying that the end user is not capable of describing desired externally observable behaviors is an extraordinarily arrogant attitude, and it is one that will be brutally punished in a capitalist marketplace.

Nobody wants to pay your company huge amounts of money to employ ivory tower design practices to build software that makes you happy.

>I believe - using your analogy - your assumption that we are nutritionalists are wrong. We ARE caterers!

Dawie,

I agree with you that we are caterers. However, the point I was trying to make is that that could be the reason why there are so many half-baked software products out there. Catering to a bunch of confused users is not going to result in something coherent, to say the least.

So, the only way out of that mess would be to start billing ourselves as 'nutritionists', claiming that we're the people who know better than our users.

>If we were trying to solve technology problems alone, you would be right. We do know more about technology than our customers.
>BUT we are trying to solve business problems. And we don't know the customer's business - he does! So we listen to him and use our technology expertise to decide how to >give him exactly what he wants.

I would be in complete agreement with you if the customers would limit themselves to talk about business only. But the problem is, they tend to meddle too much with the technology side of the problem, and at the early stages too. The thing is, during many of the inception sessions I've had with my customers, they love to kick off by telling me how they need a database with such-and-such tables, and keys, and special codes. That's the first sign of trouble. For some reason, customers feel that we should speak the same language, and they often times bend over backwards to learn my 'lingo' and to talk the database crap, web pages and mouse clicks, even UML (believe it or not).

I am always on the watchout against such atrocious behavior. I always insist that we shouldn't allow any aspect of a computer-related terminology to enter our inception and elaboration phases. Only once we reach the design phase (meaning, we're past the requirements gathering and problem domain definition phase, even past the CRC cards and the use case scenarios phase), can we open up and start talking computer-related stuff.

So, if we can get to the point where we'll be talking strictly business with the customer, I absolutely agree that the customer should be 100% in the drivers seat. But then to translate these business requirements into the particular medium of technical implementation, we shouldn't allow the customer in. Let alone listen to his/her wishes.

Alex

> As Bill noted, the requirements gathering and high-level
> design process needs to be a conversation between the
> customer and the development team.
>
> But in the end, the customer is king when it comes to
> externally observable system behaviors. For all other
> behaviors, and for implementation costs, the development
> team knows best.
>
> Saying that the end user is not capable of describing
> desired externally observable behaviors is an
> extraordinarily arrogant attitude, and it is one that will
> be brutally punished in a capitalist marketplace.
>
> Nobody wants to pay your company huge amounts of money to
> employ ivory tower design practices to build software that
> makes you happy.


What we need to establish here is that there are three things involved:

1. Business requirements gathering

2. Formulating the requirements that are needed to define end-users' interaction with the system

3. Formulating requirements needed for implementing the system

Regarding the issue number 1, the development team has no business poking their heads in there. Business people can perfectly well gather their own requirements. In other words, they don't need us, nor do they need any technology in order to accomplish that.

Step number 3, on the other hand, does not require business people at all. We, the developers, can perfectly well gather our own requirements, and figure out how to implement the system in the most efficient manner. This is where methodologies such as XP have their say.

Step number 2, however, is tricky. This is where the 'externally observable system behaviors' are defined. And that's where we must be careful not to give in to what the 'king' (i.e. the customer) wants. The core problem here is that the customers have been conditioned by years of using lousy, inadequate systems. After being abused for such a long time, they grew thinking that it's the only possible way to go. Consequently, they'll always insist on having the least optimal solutions.

It is our duty here to step in and be adamant in insisting that there is a better way. We should take it on ourselves to calmly explain to the customer that they've been abused by the half-baked, incoherent products, and that they need to expect and demand more from engineered products. Instead of interpreting this as an 'extraordinarily arrogant attitude', as you say, they will most certainly appreciate our care. Evidence (in my case, ample evidence) has shown that not only is this not being brutally punished in a capitalist marketplace, it is being richly rewarded.

It all depends on your perspective, of course, but I am nevertheless adamant that we don't have to act subservient to our confused customers in order to gain their business and their respect. Customers will respect a very knowledgeable service provider who is exhibiting genuine care for customers' true needs.

Thinking otherwise exposes extraordinarily arrogant attitude, in my opinion.

Alex

I believe that Fowler is correct (today) and incorrect (tomorrow).

In SODA (service oriented development of applications) the emphasis is reuse and flexibility.

In the past (OO & CBD days), reuse was a utopian dream that couldn't be realized. [skipping 5 reasons why] So, yes... don't bother with flexibility here...

As we move away from statically compiling applications and into orchestrating applications through loosely coupled interfaces, it becomes obvious that the services will require flexible interfaces.

I was hoping that Fowler (generally a pioneer) would begin to preach this... start letting people know that glue-code is dead, and for that matter "design for today" is dead as well. It is time to quit talking about refactoring, just-in-time-design, etc. and time to start figuring out how to orchestrate the service oriented application.
IMHO - Jeff

> Interesting comments from Martin Fowler, as usual. His
> point that the Gang of Four book is basically about the
> use of interfaces is very important. There's now at least
> one other book that emphasises the value of interfaces: my
> new book on J2EE, "Expert One-on-One J2EE Design and
> Development" has a lot to say on the merits of interfaces.
> See http://www.wrox.com/books/1861007841.htm. In
> particular, it looks at how JavaBeans can be used to
> maximise the benefits of interfaces.
>
> I think the main reason that many developers under-utilize
> interfaces is that they perceive it increases complexity.
> Through a consistent infrastructure framework such as I
> discuss in the book, and the use of JavaBeans, the
> complexity can be avoided.

Could you elaborate on how JavaBeans maximize the benefits of interfaces? Do you mean plain old JavaBeans or EJBs? How does a consistent infrastructure framework and use of JavaBeans help avoid complexity?