In one sentence, here's why: humans are notoriously bad at keeping "self" distinct from "other". Egomania, projection (transference), and enmeshment are well-known symptoms of this problem. OK, so I hear you saying, "yeah, but what does this have to do with programming?" It certainly seems absurd to suggest that if we are bad at something we know the most about (our "selves"), how could we possibly say that we have a good approach for the programming analogues - objects, modules, etc.
A recent offline conversation with Paul Hammant, co-lead of PicoContainer, (Inversion of Control/Dependency Injection) was a primary motivator for this blog entry. IoC is a simple, elegant way to encapsulate dependencies, yet according to Paul, it's taken years for the developer community to see the value in using it. So, programmers have serious issues with not seeing and removing dependencies. In light of distributed solutions, which by their very nature require considerable thought and effort in removing dependencies, and that human tendencies interfere with eliminating them, then it should be no surprise that distribution is hard.
Not only are we bad at it as humans, several other factors feed back and further discourage decoupling: it's a hard problem to solve even when it can be seen; the less it is practiced - the harder it gets to undo; and, the original team is usually gone by the time the problems become insurmountable.
What in particular makes decoupling a hard problem to solve? For one, it takes a lot of experience and a bit of natural ability to perceive software at different levels of abstraction. Without that combination of (learned) skill and (natural) ability, it is hard to see that a dependency exists AND then to remove that dependency. For another, it's human nature to take the easy/lazy route first. I get a kick out of the demotivational poster Procrastination : "Hard work often pays off after time, but laziness always pays off now."
In addition, when programmers fail to implement decoupling early, the coupling/cohesion problems start to snowball. Coupling problems tend to manifest themselves indirectly through unexpected side-effects. The first response is usually to add more code to either limit or expand functionality. Over time, this sort of coding crud piles up. Sure refactoring will help in cleaning some of these problems, however, this approach often misses "big picture" interactions. On the topic of agile, refactoring and YAGNI are great general principles. But to clarify, they in no way suggest that programmers should skip designing in loose-coupling. In fact, the first CouplingAndCohesion c2.com wiki entry by Wyatt Matthews says that loose coupling and YAGNI need to go hand-in-hand:
This dictates against PrematureGeneralization. After all, PrematureGeneralization increases an object's dependencies needlessly.
This supports DontRepeatYourself. If I am too interdependent, I've probably repeated myself needlessly and should RefactorMercilessly.
This supports OnceAndOnlyOnce. DontRepeatYourself recommends OAOO whenever possible.
This wants to support YAGNI. After all, if YouArentGonnaNeedIt, why head toward more instability?
The last form of negative selection also becomes significant over time: the original team is gone. This problem is two-fold -- first, no one is around to take responsibility for the problems, and second, a great deal of the application knowledge base left with the original team members. The first problem is particularly compounded by consulting. Most consultants are in-and-out in less than a year. My experience suggests that the most serious problems start cropping up around the one year mark. Since consultants are under heavy pressure to "deliver", why should they invest their time in the difficult and time consuming task of eliminating dependencies? Because the problems are subtle, widespread and not usually associated with high coupling, the consulting company rarely takes heat for their role in the problems. Considering that coupling issues become apparent much more quickly in a distributed system, it should come as no surprise that consultants recommend avoiding it.
The first step in finding a solution is in recognizing the source of the problems. Let's first look at the human analogues.
Enmeshment can be slowly eliminated by unweaving the co-dependencies. Listing and communicating previously unspoken expectations is a great start. Difficulties encountered when this process starts are due to being overwhelmed by the sheer number of unspoken expectations, and being surprised by the fact that they were obliviously unaware of most of those expectations.
In some ways, marriage vows do a disservice to the richer subtleties in intimate human interaction. Namely, two people don't come together to become one, they come together to become three! There will always be the self and the other. The third ingredient is the relationship itself. Unless the relationship is tended to by both people, it won't work. The most successful couples:
address expectations through rituals (protocols) and communication
The second step in solving problems is to discover and leverage the lessons others have learned. The following excerpts only touch the tip of the iceberg. Read, study, and internalize these references - make them your own.
Carlos Perez' wrote recently in his Manageability blog about six operators of modularity that are essential for modular systems:
Splitting - Modules can be made independent.
Substituting - Modules can be substituted and interchanged.
Excluding - Existing Modules can be removed to build a usable solution.
Augmenting - New Modules can be added to create new solutions.
Inverting - The hierarchical dependencies between Modules can be rearranged.
Porting - Modules can be applied to different contexts.
The context in which one uses a component includes the other components with which it communicates. If a component makes assumptions about how those components are implemented, it becomes hard to reuse in combination with different components.
Problem
How can you reduce a component's dependence on other components in its context?
Solution
Define protocols by which components interact separately from the components themselves.
Codify these protocols as abstract interfaces
Implement components that rely only on abstract interfaces. That is, components should refer only to abstract interfaces not to concrete classes that conform to the interfaces.
For the final reference, all I can say is "Wow". This paper contains so much good stuff that it should be included in every developer's library. Read Caterpillar's Fate on the c2.com Wiki to see what I mean.
Hopefully it isn't too difficult to see that people are bad at recognizing differences between self and others. It should also not be too hard to see that these difficulties would spill over into programming.
We should also recognize that other factors aggravate the problem and further discourage decoupling. One, it's a hard problem to see, let alone, to solve. Two, if developers don't keep on top of, and continually remove, dependencies, the harder it gets to remove them. Third, these problems are exacerbated by the fact that the original team (and especially consultants) leave the project taking hard-learned domain expertise with them sometimes to avoid the negative consequences of blame.
Distributed applications are particularly hard-hit by these things. Distributed applications are inherently more difficult than "standard" applications because they simply cannot be built without making them very decoupled. (See Carlos' coupling table above).
On the positive side, many very bright people have seen solutions to overcoming these dependency problems. The solutions are subtle and complex which means that they can't be applied in cookie-cutter fashion. If developers assimilate these solutions into their personal knowledge base then all types of applications can be taken to the next level of complexity.
A type of defence mechanism. A person experiences an emotion or thought that they can't deal with exactly for whatever reason. The unacceptable feeling or thought is experienced as though someone else had been thinking or feeling it.
The term "Enmeshment" comes from the family systems theory tradition. Enmeshment refers to a condition where two or more people weave their lives and identities around one another so tightly that it is difficult for any one of them to function independently. The opposite extreme way of relating, Detachment, refers to a condition where the people are so independent in their functioning that it is difficult to figure out how they are related to one another. Healthy relationships are thought to be described by the space between enmeshment and detachment.
I agree. All of those things are tough. They are hard for people to see and act upon.
The one thing that keeps me optimistic is noticing that when people start to see some advantage to creating classes in test harnesses through TDD, they start to create loosely coupled software by default and they start to notice what it looks like and how it is different from the intermingled stuff.
Sometimes that is more effective than telling people. The thing that is hard to face is that many people are not as facile with abstraction as others and they have to approach this kind of learning through experience.
Also, I think the causes of your argument, "because achieving consensus is equally hard," can be reduced to a combination of the three problems: Egocentrism, Projection, and Enmeshment. Tying these problems with "Unskilled and Unaware of it" (http://www.apa.org/journals/psp/psp7761121.html), makes designing by consensus an effort in frustration and futility. That leads to the next equally hard problem of deciding "who". That's one I'm still thinking about.
IANAP (I am not a Psychologist), however you may need to go beyond individual psychology and into group psychology to analyze why people can't get agreement. However, I think we can all come up with plenty of motivations why cooperation isn't a good thing (e.g. employment security, resume padding, empire building etc.)
Previous Topic
