> The important part, as Ward emphasized, is that you have
> to be clear early on what is fundamental (i.e. static) and
> what should be left sketchy or omitted (dynamic).

In other words, you're saying...
1) You cannot get away from doing design up front and predicting what changes are permitted and what aren't.
2) There are limitations on what you can change later in the project.

Unfortunately, both these points strongly contradict "Agile" theory which says that:
1) Don't attempt to predict and design for possible later changes.
2) Anything can be changed at anytime by anyone.

Vince.

Not all requirements changes are of the type where you "write 20 statements and change four". In my experience, there are many types of decisions that are going to be much cheaper to make earlier than later--regardless of your choice of process, methodology, programming language, or hairstyle.

Globalization/internationalization
Exception handling and logging policy
"Branded" versions of apps or webapps
Clustered deployment
Client type (i.e. adding a Swing client to a webapp)

IMO you'd be crazy not to plan for internationalization if there's even a 25% chance that a Japanese port is in your future--even if the only action you take is to pick a programming platform that supports Unicode. Yet the XP literature unequivocally states "Turn a blind eye towards future requirements and extra flexibility." http://www.extremeprogramming.org/rules/early.html

It seems to me that this particular XP rule makes a lot of sense where you're talking about programming at a micro level--individual implementations of classes or algorithms, or even of a module/package--but less so when you're talking about high-level architecure or application-wide concerns.

How do you know what will not change?

Unless you contend that any piece of software can morph into any other piece of software, then the initial design must include constraints. The trick is to make the initial constraints no more limiting than is absolutely necessary. Call it art or engineering, it’s part of the responsibility of the designer to make this call.

Parnas was looking at this from the opposite end: decide what may change and encapsulate it. The problem, as you say, is that we can’t know what changes will be required. That is why there is an advantage in deciding what will is absolutely fundamental and encapsulating everything else.

Is this pure semantics? I don’t think so. The software that has lasted longest has been that where the original designers displayed genius (or luck) in creating a kernel, which is fundamentally separate from its visible functions and which did no more than absolutely necessary to define it as a product.

I’m certainly suggesting that it’s not possible (except fortuitously) to create software that is entirely malleable. One has to define a core product, which at some point in time will have to be discarded.

"The software that has lasted longest has been that where the original designers displayed genius (or luck) in creating a kernel."

Yes, and the CPU architecture has not evolved in any dramatic way either. It is ludicrous to compare CPU and kernel software that is generated using documentation that will make a phone book look like a joke to business software that is usually run by business owners who are clueless about s/w development.

In my opinion there are two types of changes:
a- Changes that are unforseen
b- Changes due to misinterpretation.

For (b), the designer failed to get the user a feel for the system as soon as possible, misread the requirement or some such "error" occured. For (a), it is usually due to circumstances that are beyond anyone's control. The pressure in (a) is because everyone has this notion that s/w is very flexible and hence anything can be changed. The reality is that software is dangerously flexible - can easily code the wrong thing and change is expensive.

In most cases, change late in the game is costly not because you have more lines of code to change but because:
- You have to update the requirements spec
- You have to update the design document
- You have to update the code
- You have to update the unit tests
- You have to update the test scripts
- You have to typically run regression tests all over again
- You have to re-educate your client
- You have to update the user's guide
- etc.

[/i]Parnas was looking at this from the opposite end: decide what may change and encapsulate it ... there is an advantage in deciding what is absolutely fundamental and encapsulating everything else.<br>
Is this pure semantics? I don’t think so.[/i]

These seem to be duals: identify what we're unsure about and encapsulate it; indentify what we're sure about and encapsulate everything else.

Seemed like the article suggested something quite different - we shouldn't worry about future changes (we don't guess right often enough). Instead design for the requirements that exist.

Seemed like the article suggested something quite different - we shouldn't worry about future changes

Now we're decidedly into shades of grey -- and I don't want to guess (keep guessing?) what somebody else is thinking.

This is probably where we should stop trying to create a rule to end all rules, and create different approaches depending on whether we're talking about some trivial one-off project clearly delineated by a particular customer, or something grander, expected to last much longer.

Maimonides: In the beginning we must simplify the subject, thus unavoidably falsifying it, and later we must sophisticate away the falsely simple beginning.

I agree with Ward that a lot of what we consider great design is often a product of evolution, rather than someone sitting down and figuring everything out up-front. I used to start designing things by first defining the interface of some component, and then figuring out how the desired components interact via their interfaces. While I still think there is room for that sort of approach, I now tend to build things by creating small pieces of functionality, and then refactoring them according to some logical criteria that often emerges during the process. I suppose that that latter approach is similar to the molding of clay, or of marble, that Ward talks about.

While at first I didn't like that latter approach on intellectual grounds (I considered it "hacking" vs. architecting), the "molding the clay" approach proves more effective than the "top-down" architecting approach in many situations. Interestingly, and counter to my initial intuition, the upfront architecting approach seems to work well on smaller projects. As the size of the project grows, and with it the number of requirements and classes, having a clear-cut, initially well thought-out architecture often becomes a burden. At some point, the decision must be made to either completely revise the up-front design (which makes that design less up-front), or just simply go without a grand design.

In lieu of a grand design, what I often find useful, though, are idioms. These are not really patterns in the fancy sense of the word, but just small habitual ways of doing things. If an idiom works well, reusing that idiom over and over again helps give the program a sense of unity, since even a large program can often be reduced to a handful of idioms. The set of idioms give the program a sense of Gestalt. Also, developers reading code based on idioms have to understand only those handful of idioms to start contributing to the program.

create different approaches
Looking at the problem and then putting together a process for that problem does seem like a smart thing to do.

The idea of "blending process models" was explicit in "Strategies for Software Engineering" Martyn A. Ould 1990.

AFAIK only Michael Jackson has tried to characterize problems (rather than solutions).

http://www.ferg.org/pfa/

> AFAIK only Michael Jackson has tried to characterize
> problems (rather than solutions).
Better references:
http://dspace.dial.pipex.com/jacksonma/

> I agree with Ward that a lot of what we consider great
> design is often a product of evolution, rather than
> someone sitting down and figuring everything out up-front.
...
> In lieu of a grand design, what I often find useful,
> though, are idioms.

This is exactly what I use. I have particular patterns of overall system architecture that I utilize in particular situations.

I often cast these patterns into classes at some point and provide enough augmentation via abstract interfaces to make them a general solution to a particular class or problem.

Frank, you sparked me to ramble some more about languages over in my blog...

http://www.artima.com/weblogs/viewpost.jsp?thread=28518

a lot of what we consider great design is often a product of evolution, rather than someone sitting down and figuring everything out up-front

This makes upfront design sound like immaculate conception! Prototyping and iteration are commonplace in design activities.

Why do we have to choose between everything upfront and nothing upfront?

first defining the interface of some component, and then figuring out how the desired components interact via their interfaces... creating small pieces of functionality, and then refactoring them according to some logical criteria that often emerges during the process

Isn't this top-down vs. bottom-up?
What about middle-out edge-to-edge design?

In "Software Requirements & Specifications" Michael Jackson points out that for inventing or designing new things top-down is the riskiest possible ordering of decisions. The largest decision, the subdivision of the whole problem, is taken first when nothing is yet known and everything remains to be discovered.

(Dijkstra sidesteps this "I assume the programmer's genius matched to the difficulty of his problem and assume that he has arrived at a suitable subdivision of the task."!)

Prototyping and iteration are commonplace in design activities - the interesting question is where are the cheap places in-the-process to do design iteration? When is design iteration waste?

As the size of the project grows, and with it the number of requirements and classes, having a clear-cut, initially well thought-out architecture often becomes a burden
How much is this due to lack of tool support?

At some point, the decision must be made to either completely revise the up-front design (which makes that design less up-front)
Wouldn't that be the (no less) up-front design of version 2?