Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
Flexibility and Complexity
November 25, 2002 7:15 AM
|
Artima.com has published Part IV of an interview in which Martin Fowler discusses design decay, the cost of flexibility and reusability, four criteria for a simple system, and test-first design of interfaces.---xj40dkcfea73---Artima.com has published Part IV of an interview in which Martin Fowler discusses design decay, the cost of flexibility and reusability, four criteria for a simple system, and test-first design of interfaces.
http://www.artima.com/intv/flexplex.html The cost of flexibility is complexity. Every time you put extra stuff into your code to make it more flexible, you are usually adding more complexity. If your guess about the flexibility needs of your software is correct, then you are ahead of the game. You've gained. But if you get it wrong, you've only added complexity that makes it more difficult to change your software. You're obviously not getting the payback. It's not hard to guess wrong about flexibility needs. You can guess wrong if requirements change. What you think is a requirement for flexibility now may go away or change in the future. You can also guess wrong if you put extra code into the program to improve flexibility but you don't get it quite right. You get more complexity without getting the flexibility you were after. The alternative is to use the XP approach and not put the flexibility in at all. XP says, since most of the time we get it wrong, just don't put the flexibility in there. Now if you can't evolve your design safely, then that is a foolish route to take. But if you can evolve your design safely, it becomes quite a nice approach. In fact it becomes a self-reinforcing approach. If you strive to keep your design as simple as possible by avoiding speculative flexibility, then it's easier to change the code because you have less complication to deal with. The code is easier to understand and easier to change. As a result, you can make changes much more quickly. What do you think of Martin Fowler's comments? |
Posts: 1 / Nickname: clamprecht / Registered: November 23, 2002 7:13 AM
Re: Flexibility and Complexity
November 25, 2002 4:52 PM
|
I've been reading a lot about XP lately. I'd like to know if the XP beliefs surrounding design apply to a database as well. For example, say I'm designing some app that involves a decent-sized database. Should the same principles apply to my database design as apply to my code design? Changing code later is one thing, but changing the database may be a different story.
Maybe you could ask Martin Fowler this for the next interview in the series -- if refactoring applies to designing a database as well. |
Posts: 12 / Nickname: carfield / Registered: September 16, 2002 3:19 PM
Re: Flexibility and Complexity
November 26, 2002 8:57 AM
|
actually there already have some discussions about this topic, but probably not easy to put in practice
http://www.agiledata.org/essays/databaseRefactoring.html |
Posts: 9 / Nickname: mfowler / Registered: November 27, 2002 3:51 AM
Re: Flexibility and Complexity
November 27, 2002 8:54 AM
|
> I've been reading a lot about XP lately. I'd like to know
> if the XP beliefs surrounding design apply to a database > as well. For example, say I'm designing some app that > involves a decent-sized database. Should the same > principles apply to my database design as apply to my code > design? Changing code later is one thing, but changing > the database may be a different story. The short answer is that you can build up a database design in an evolutionary way as well. My colleague Pramod Sadalage is starting to write up how you can do this based on his experiences on a sizable project. He's started a discussion group at http://groups.yahoo.com/group/agileDatabases . I'm hoping to get an article on the subject on my web site soon |
Posts: 12 / Nickname: carfield / Registered: September 16, 2002 3:19 PM
Re: Flexibility and Complexity
November 27, 2002 10:01 AM
|
The discussion of "Flexibility and Complexity" and "Flexible versus Reusable" is a very good explanation of how to have flexibility code with simple design, which XP suggesting. But it is hard for me to practice, since I touch computer programming, what I learn is put more time to design, think about the flexibility and reusability.
|
Posts: 5 / Nickname: alexbcit / Registered: November 28, 2002 3:48 PM
Re: Flexibility and Complexity
November 28, 2002 9:39 PM
|
As a long time practitioner of the "quality before design" discipline, I have a serious problem with XP's insistence on 'emerging' design. The differences between XP and the quality-before-design (QBD for short) schools are fundamental and irreconcillable. I've reached this conclusion after several months of intense debate with several keen practitioners of XP.
At this time, I'd like to know what are XP's "high priesthood", such as Fowler et al, doing to address the objections posed to them by the QBD camp? To refresh your memory, what we (the QBD-ers) are big on is the fact that, rather than view our profession as being akin to caterers, that is, catering to our customers' whims, we prefer to view ourselves as being akin to nutrition specialists. That means that we're telling our customers something along these lines: "Of course we realize that you know what is it that *you like*. However, we think that you may not always know what is it that's *good for you*. We are the experts that specialize in knowing what's good for you." The above is similar to how nutritionists in general operate -- they do not observe what is it that their patients like to eat. Instead, they prescribe the diets that are the healthiest choice for the given patient. In the process, they totally ignore any objections that may occur. In my conversation with XP proponents, they all seemed to assume the stance of the caterers -- we'll deliver whatever it is that the customer desires. They seem devoted to being extremely sensitive to their customers' wishes. I think that's utterly wrong, because the customers in most cases have develped notoriously bad, if not right down dangerous habits, and catering to those habits equates to severely damaging their wellbeing. We, the information services specialists, must take the initiative to ensure that our customers get the highest quality possible. In order to achieve that, we must focus solely on the quality (whatever 'quality' may mean in a given situation). Only once the quality has been established, do we allow the process to move on to the design (that is, elaboration) and construction phase. At which point we let other sets of criteria govern the remainder of the process (as it gets largely irrelevant to the quality). In other words, we don't fret so much over the quality of the code per se. We worry about the quality of the intentions as implemented by the code. And we don't believe that that quality could be governed by the code itself, nor by the manner in which that code was produced. It's similar to delivering a new car to the customer -- what's important is that the car satisfies the criteria for highest possible quality. What methodology was used when building that car is of secondary importance. The quality of that car was unambigously established long before anyone put any design efforts on paper. Or else the car would end up being dysfunctional. Alex |
Posts: 1 / Nickname: dfstrauss / Registered: November 28, 2002 7:21 PM
Re: Flexibility and Complexity
November 29, 2002 0:37 AM
|
Alex,
I believe - using your analogy - your assumption that we are nutritionalists are wrong. We ARE caterers! If we were trying to solve technology problems alone, you would be right. We do know more about technology than our customers. BUT we are trying to solve business problems. And we don't know the customer's business - he does! So we listen to him and use our technology expertise to decide how to give him exactly what he wants. |
Posts: 5 / Nickname: alexbcit / Registered: November 28, 2002 3:48 PM
Re: Flexibility and Complexity
November 29, 2002 11:41 AM
|
>I believe - using your analogy - your assumption that we are nutritionalists are wrong. We ARE caterers!
Dawie, I agree with you that we are caterers. However, the point I was trying to make is that that could be the reason why there are so many half-baked software products out there. Catering to a bunch of confused users is not going to result in something coherent, to say the least. So, the only way out of that mess would be to start billing ourselves as 'nutritionists', claiming that we're the people who know better than our users. >If we were trying to solve technology problems alone, you would be right. We do know more about technology than our customers. >BUT we are trying to solve business problems. And we don't know the customer's business - he does! So we listen to him and use our technology expertise to decide how to >give him exactly what he wants. I would be in complete agreement with you if the customers would limit themselves to talk about business only. But the problem is, they tend to meddle too much with the technology side of the problem, and at the early stages too. The thing is, during many of the inception sessions I've had with my customers, they love to kick off by telling me how they need a database with such-and-such tables, and keys, and special codes. That's the first sign of trouble. For some reason, customers feel that we should speak the same language, and they often times bend over backwards to learn my 'lingo' and to talk the database crap, web pages and mouse clicks, even UML (believe it or not). I am always on the watchout against such atrocious behavior. I always insist that we shouldn't allow any aspect of a computer-related terminology to enter our inception and elaboration phases. Only once we reach the design phase (meaning, we're past the requirements gathering and problem domain definition phase, even past the CRC cards and the use case scenarios phase), can we open up and start talking computer-related stuff. So, if we can get to the point where we'll be talking strictly business with the customer, I absolutely agree that the customer should be 100% in the drivers seat. But then to translate these business requirements into the particular medium of technical implementation, we shouldn't allow the customer in. Let alone listen to his/her wishes. Alex |
Posts: 1 / Nickname: corby / Registered: November 29, 2002 4:36 AM
Re: Flexibility and Complexity
November 29, 2002 9:45 AM
|
As Bill noted, the requirements gathering and high-level design process needs to be a conversation between the customer and the development team.
But in the end, the customer is king when it comes to externally observable system behaviors. For all other behaviors, and for implementation costs, the development team knows best. Saying that the end user is not capable of describing desired externally observable behaviors is an extraordinarily arrogant attitude, and it is one that will be brutally punished in a capitalist marketplace. Nobody wants to pay your company huge amounts of money to employ ivory tower design practices to build software that makes you happy. |
Posts: 5 / Nickname: alexbcit / Registered: November 28, 2002 3:48 PM
Re: Flexibility and Complexity
November 29, 2002 3:16 PM
|
> As Bill noted, the requirements gathering and high-level
> design process needs to be a conversation between the > customer and the development team. > > But in the end, the customer is king when it comes to > externally observable system behaviors. For all other > behaviors, and for implementation costs, the development > team knows best. > > Saying that the end user is not capable of describing > desired externally observable behaviors is an > extraordinarily arrogant attitude, and it is one that will > be brutally punished in a capitalist marketplace. > > Nobody wants to pay your company huge amounts of money to > employ ivory tower design practices to build software that > makes you happy. What we need to establish here is that there are three things involved: 1. Business requirements gathering 2. Formulating the requirements that are needed to define end-users' interaction with the system 3. Formulating requirements needed for implementing the system Regarding the issue number 1, the development team has no business poking their heads in there. Business people can perfectly well gather their own requirements. In other words, they don't need us, nor do they need any technology in order to accomplish that. Step number 3, on the other hand, does not require business people at all. We, the developers, can perfectly well gather our own requirements, and figure out how to implement the system in the most efficient manner. This is where methodologies such as XP have their say. Step number 2, however, is tricky. This is where the 'externally observable system behaviors' are defined. And that's where we must be careful not to give in to what the 'king' (i.e. the customer) wants. The core problem here is that the customers have been conditioned by years of using lousy, inadequate systems. After being abused for such a long time, they grew thinking that it's the only possible way to go. Consequently, they'll always insist on having the least optimal solutions. It is our duty here to step in and be adamant in insisting that there is a better way. We should take it on ourselves to calmly explain to the customer that they've been abused by the half-baked, incoherent products, and that they need to expect and demand more from engineered products. Instead of interpreting this as an 'extraordinarily arrogant attitude', as you say, they will most certainly appreciate our care. Evidence (in my case, ample evidence) has shown that not only is this not being brutally punished in a capitalist marketplace, it is being richly rewarded. It all depends on your perspective, of course, but I am nevertheless adamant that we don't have to act subservient to our confused customers in order to gain their business and their respect. Customers will respect a very knowledgeable service provider who is exhibiting genuine care for customers' true needs. Thinking otherwise exposes extraordinarily arrogant attitude, in my opinion. Alex |
Posts: 2 / Nickname: jeffs / Registered: November 29, 2002 1:27 PM
Re: Flexibility and Complexity
November 29, 2002 6:50 PM
|
I believe that Fowler is correct (today) and incorrect (tomorrow).
In SODA (service oriented development of applications) the emphasis is reuse and flexibility. In the past (OO & CBD days), reuse was a utopian dream that couldn't be realized. [skipping 5 reasons why] So, yes... don't bother with flexibility here... As we move away from statically compiling applications and into orchestrating applications through loosely coupled interfaces, it becomes obvious that the services will require flexible interfaces. I was hoping that Fowler (generally a pioneer) would begin to preach this... start letting people know that glue-code is dead, and for that matter "design for today" is dead as well. It is time to quit talking about refactoring, just-in-time-design, etc. and time to start figuring out how to orchestrate the service oriented application. IMHO - Jeff |
Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
Re: Flexibility and Complexity
November 30, 2002 2:58 PM
|
> I believe that Fowler is correct (today) and incorrect
> (tomorrow). > > In SODA (service oriented development of applications) the > emphasis is reuse and flexibility. > > In the past (OO & CBD days), reuse was a utopian dream > that couldn't be realized. [skipping 5 reasons why] So, > yes... don't bother with flexibility here... > > As we move away from statically compiling applications and > into orchestrating applications through loosely coupled > interfaces, it becomes obvious that the services will > require flexible interfaces. > I think it is a question of extent. To what extent do you make your interface flexible? Just because a service interface is often a published interface, one which you can't refactor because you can't fix all client code married to the interface, doesn't mean you don't still have to balance simplicity with flexibility. I think developers should be conservative about the functionality they put into any interfaces, including published ones, because it is hard to predict the future. It is hard to know what flexibility will actually be needed in the future. One thing you can do with Jini service interfaces that I'm not yet sure how easy it is to do with web service interfaces is evolve them over time. You can make all sorts of binary compatible changes to Jini service interfaces that add functionality without breaking existing clients. One of the articles I want to publish on this site next year will be an exploration of how to design web services interfaces so they can be evolved. But in the world of Jini, at least, you can refactor published service interfaces. You are not as free as when the interface is not published, but there are all kinds of binary compatible changes you can make without breaking client code. I also want to publish a web services article next year about contracts, because I believe thinking of interfaces as contracts is an essential part of designing published service interfaces that are changable over time. |
Posts: 2 / Nickname: jeffs / Registered: November 29, 2002 1:27 PM
Re: Flexibility and Complexity
November 30, 2002 6:29 PM
|
Bill - all good points.
You are correct. Just because you can design for the future doesn't mean you have to. And you are correct again, it is a matter of 'extents'. I would suggest that the extent would be based on the reusability of the platform. Creating applications that allow for reusability through neutralizers (OS, network, platform, language, location) incents one to create reusable code and thus consider designing for reuse. Service Oriented Architectures do this. IMHO, we have a major shift in how applications will be written. My issue is that the gurus of OO seem to be stuck in OO-Land. When we talk about 'design' issues we must now preface them with an era (Structured, Object Oriented, Service Oriented) - because the rules have changed! Jeff |
Posts: 5 / Nickname: alexbcit / Registered: November 28, 2002 3:48 PM
Re: Flexibility and Complexity
November 28, 2002 9:38 PM
|
As a long time practitioner of the "quality before design" discipline, I have a serious problem with XP's insistence on 'emerging' design. The differences between XP and the quality-before-design (QBD for short) schools are fundamental and irreconcillable. I've reached this conclusion after several months of intense debate with several keen practitioners of XP.
At this time, I'd like to know what are XP's "high priesthood", such as Fowler et al, doing to address the objections posed to them by the QBD camp? To refresh your memory, what we (the QBD-ers) are big on is the fact that, rather than view our profession as being akin to caterers, that is, catering to our customers' whims, we prefer to view ourselves as being akin to nutrition specialists. That means that we're telling our customers something along these lines: "Of course we realize that you know what is it that *you like*. However, we think that you may not always know what is it that's *good for you*. We are the experts that specialize in knowing what's good for you." The above is similar to how nutritionists in general operate -- they do not observe what is it that their patients like to eat. Instead, they prescribe the diets that are the healthiest choice for the given patient. In the process, they totally ignore any objections that may occur. In my conversation with XP proponents, they all seemed to assume the stance of the caterers -- we'll deliver whatever it is that the customer desires. They seem devoted to being extremely sensitive to their customers' wishes. I think that's utterly wrong, because the customers in most cases have develped notoriously bad, if not right down dangerous habits, and catering to those habits equates to severely damaging their wellbeing. We, the information services specialists, must take the initiative to ensure that our customers get the highest quality possible. In order to achieve that, we must focus solely on the quality (whatever 'quality' may mean in a given situation). Only once the quality has been established, do we allow the process to move on to the design (that is, elaboration) and construction phase. At which point we let other sets of criteria govern the remainder of the process (as it gets largely irrelevant to the quality). In other words, we don't fret so much over the quality of the code per se. We worry about the quality of the intentions as implemented by the code. And we don't believe that that quality could be governed by the code itself, nor by the manner in which that code was produced. It's similar to delivering a new car to the customer -- what's important is that the car satisfies the criteria for highest possible quality. What methodology was used when building that car is of secondary importance. The quality of that car was unambigously established long before anyone put any design efforts on paper. Or else the car would end up being dysfunctional. Alex |
Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
Re: Flexibility and Complexity
November 29, 2002 7:54 AM
|
> To refresh your memory, what we (the QBD-ers) are big on
> is the fact that, rather than view our profession as being > akin to caterers, that is, catering to our customers' > whims, we prefer to view ourselves as being akin to > nutrition specialists. That means that we're telling our > customers something along these lines: "Of course we > realize that you know what is it that *you like*. However, > we think that you may not always know what is it that's > *good for you*. We are the experts that specialize in > knowing what's good for you." > This paragraph reminded me of Ken Arnold's discussion of the arrogance of taste, which is on this page: http://www.artima.com/intv/taste.html I have found the process of getting requirements out of customers has mostly to do with human relationships and little to do with technology. In my experience clients don't usually know exactly what they want, but they usually have general ideas about what they want. Also, different people usually have different pet features they understand in more detail that they want included in the requirements. Often the client doesn't know as much about technology as I do, but they know far more about their business than I do. I see my role as a technical consultant as helping them figure out how best to achieve their business goals given the current technology. What I find helpful is bringing all the parties together who have an opinion and want to feel included in defining the requirements, and get everybody talking about the requirements. I explore the requirements space together and help them figure out what they want. It is a process of exploration and negotiation between all the parties in the meeting. I do make suggestions and propose ideas, but for the most part the process feels like listening. After such a meeting I'll write down what I think we all agreed to and send it to everybody who attended the meeting. My intuition is that it might make sense for developers to see themselves as nutritionists in the area of technology, but not in the area of the client's or employer's business. |
Posts: 5 / Nickname: alexbcit / Registered: November 28, 2002 3:48 PM
Re: Flexibility and Complexity
December 2, 2002 4:50 PM
|
>This paragraph reminded me of Ken Arnold's discussion of >the arrogance of taste, which is on this page:
> >http://www.artima.com/intv/taste.html I believe Ken is absolutely right, but I'm not sure why he put it in such lukewarm terms. In other words, things need to be spelled out more strongly. For example, I disagree with his use of the 'painting the house' analogy. I think it's weak and doesn't serve the purpose well. A much better example would be building a house: yes, our customers may have all kinds of preferences when telling us how they want us to build their house, but there are many rock-bottom restrictions which no one's wish should be able to violate. I don't care if the customer promises to pay me extra $100,000.00, if the thing he's asking for is going to endanger the whole project, I'm going to decline him flatly. I insist that we must adopt the same rigorous approach when designing a software product. Yes, the customers will quite likely demand foolish things from a product we're supposed to deliver, but we know better, and we should straighten them out as soon as the foolish demand arises. Some people may interpret that as being arrogant -- I interpret it as being a professional. >What I find helpful is bringing all the parties together >who have an opinion and want to feel included in defining >the requirements, and get everybody talking about the >requirements. I explore the requirements space together >and help them figure out what they want. It is a process >of exploration and negotiation between all the parties in >the meeting. I do make suggestions and propose ideas, but >for the most part the process feels like listening. After >such a meeting I'll write down what I think we all agreed >to and send it to everybody who attended the meeting. I used to practise exactly the same apporach. Then one day I realized that it has the tendency to muddy the issue. What I do now is tell my customers not to involve me until they are absolutely certain what is it that they want to accomplish from the business point of view. That means, they should be able to sit down and hammer out their business requirements on their own, without having to involve computers/software at any point. I give them instructions to imagine they are dealing with the Alladin's magic lamp, so that whenever they feel they have a need for some sort of information service, they just put down something like: 'rub the lamp, the genie comes out, and we tell him what to do'. Everything else should be pure business, nothing to do with the genie, nothing to do with computers. Then, once they are clear on that, I jump in and address the genie-related issues. I make an assessment as to whether the wishes expressed in there are reasonable, meaning whether the techno-genie can grant them or not. Most of the time, he can. But *how* is techno-genie going to fulfill their wishes is really none of their business. They only worry about *what* is it they'd like to see as the outcome/externally perceived behavior of the system. >My intuition is that it might make sense for developers to >see themselves as nutritionists in the area of technology, >but not in the area of the client's or employer's business. Amen. Alex |
Posts: 1 / Nickname: rodj / Registered: November 25, 2002 9:30 PM
Re: Flexibility and Complexity
November 26, 2002 2:36 AM
|
Interesting comments from Martin Fowler, as usual. His point that the Gang of Four book is basically about the use of interfaces is very important. There's now at least one other book that emphasises the value of interfaces: my new book on J2EE, "Expert One-on-One J2EE Design and Development" has a lot to say on the merits of interfaces. See http://www.wrox.com/books/1861007841.htm. In particular, it looks at how JavaBeans can be used to maximise the benefits of interfaces.
I think the main reason that many developers under-utilize interfaces is that they perceive it increases complexity. Through a consistent infrastructure framework such as I discuss in the book, and the use of JavaBeans, the complexity can be avoided. |
Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
Re: Flexibility and Complexity
November 30, 2002 2:04 PM
|
> Interesting comments from Martin Fowler, as usual. His
> point that the Gang of Four book is basically about the > use of interfaces is very important. There's now at least > one other book that emphasises the value of interfaces: my > new book on J2EE, "Expert One-on-One J2EE Design and > Development" has a lot to say on the merits of interfaces. > See http://www.wrox.com/books/1861007841.htm. In > particular, it looks at how JavaBeans can be used to > maximise the benefits of interfaces. > > I think the main reason that many developers under-utilize > interfaces is that they perceive it increases complexity. > Through a consistent infrastructure framework such as I > discuss in the book, and the use of JavaBeans, the > complexity can be avoided. Could you elaborate on how JavaBeans maximize the benefits of interfaces? Do you mean plain old JavaBeans or EJBs? How does a consistent infrastructure framework and use of JavaBeans help avoid complexity? |
Posts: 1 / Nickname: caminante / Registered: December 4, 2002 4:36 PM
Re: Flexibility/Complexity and Interfaces
December 4, 2002 10:12 PM
|
Having created many systems based on interfaces and factories, I've come to the conclusion that they are not always appropriate. I see definite a logical link between the flexibility/complexity tradeoff and the use of interfaces. The unconstrained use of interfaces - even when there is no foreseeable future need to ever have more than one concrete implementation - adds non-negligible complexity: extra names and potentially a factory. I would venture to claim that the use of an interface should be evaluated by the same flexibility/complexity criteria as any other issue.
Furthermore, there are certainly times where the use of an abstract base class is desirable - either as the root or as an implementation of the interface that the concrete implementations can extend. There's an interesting item by the JDOM authors (Jason Hunter and Brett McLaughlin) called: "Why is the JDOM API defined in terms of concrete classes rather than interfaces?" See http://jdom.org/docs/faq.html An excerpt: "This issue has been discussed several times on the JDOM mailing list, with several people on both sides. In general, many people feel that a class-based API is better when subclassing is not needed, while an interface-based API is better when subclassing is needed. However, either system can be used in either case. We started with interfaces actually. During our pre-release review to some peers we received the feedback we should try concrete classes. We did, and the design was much better for it." |