Summary
Jello has several very nice properties. It's cheap. It tastes good. It comes in cool colors. It's firm but not stiff. Systems need to be built like that. Firm, but not stiff. With clear boundaries.
Advertisement
I have been, more or less, uncomfortable at work since nearly my first day. We use an agile process and this means, among other things, there is a brief daily meeting where everyone presents what they did in the last 24 hours and what they plan to accomplish in the next 24. Frequently I've been simply dumbfounded by how much people actually do in 24 hours. Finished this, added that, fixed such and so. When my turn comes I find myself mumbling things like "trying to figure out how XYZ works", or "working on understanding where to put ABC". Every few days, more or less, I get to say "finished". It's difficult to describe how uncomfortable how dumb I sometimes feel in the face of the endless series of "finished"s, "done"s, "next"s that spout from the mouths of folks at pace that is, simply, incredible. So, uncomfortable. Until a couple of days ago.
I had to implement something in a part of the system that I've never ventured into (UI stuff, if you're curious). I was lucky enough to be able to convince this incredibly bright kid to pair with me and help guide me through the UI code. What this really means of course is that he drove and, mostly, I watched. At some point we needed something that was not available in the system in an immediate and obvious way. There was something that sort of did what we wanted, but not exactly. So, without a thought, he snarfed up the code, put it close to where we were and made it do what we wanted. Voila! Done! And I realized a couple of things. First, I now knew why I was slower in "completing" things than it seemed others were. Second, this is why XP doesn't scale.
It has been my contention for some time that XP doesn't scale well in terms of organization size. That is, XP works very, very well when the people working on the system have essentially total system knowledge. However there comes a point in the life of non-trivial systems where it is impossible for any one human to understand the system in the nuanced, detailed way that one must in order to not end up create a pile of spaghetti.
The code we copied and modified to do what we needed came from a major subsystem. What we did not do is consider what the impact to the system, and other developers, this semantic decentralization caused. In essence what we did was create a slightly varying way of doing something that was very specific to what we needed. So if you want to do X, you would look here. If you want to do X', look there. If you don't know that X or X' exists, you'll probably just create your own. The likelihood that you'll know these exists is probably proportional to how hard you have to work learn where it is. Knowing where something is, or where it should go, in a larger system is what I call the vocabulary of the system. I use this term in the Alexander sense.
This, in effect, is why I say "Done" in our daily meetings rather less frequently than others. When I need to add or modify something in the system my first order concern is not to get the damned thing functionally complete and move on. No, my first order concern is to not modify the vocabulary of the system without knowing I'm doing so. That is, I spend real time understanding where something should go, why it should go there, and what the impact on everyone else is. I think this is valuable and required. It allows one to reason about the system. If the ABC thing is responsible for, say, describing the structure of the persistent store, then when one needs to understand some aspect of the structure of the persistent store they look at ABC. If it's not there, it's not in the system and you add it. To ABC. Not to XYZ. This avoids something that I think is far, far worse than duplicated code, something XP says is very bad. It avoids duplicated ideas.
I think the folks who think about XP need to start thinking about how to address the issue of larger systems and their attendant organizations. How does one ensure that ideas are not duplicated in the system? I reject the idea, entirely, that when one finds these duplications you refactor them out with vigor. It sounds like a great idea, but it ignores the real world of deadlines, making money, and the expectations of business people. The fact is, and I can say this from real world experience, when you find these sorts of duplications you almost never have time to remove them entirely. Other commitments have been made, your time has been allocated in some other fashion, you have a set of user visible functionality that has been promised. And you must deliver in a bounded time frame. The number of times the business can tolerate you saying, "Right, found this duplication and I have to remove it. I need to add 20% to my estimate" is substantially smaller than the number of times you need to say it.
XP doesn't scale. Believe it. I'm living it and it's true. XP is, I think, the right substrate. It's got the ring of truth to it. But it needs something else. Something above it that talks about how to rationalize a change in the context of the longevity of the system. It has to talk about not just refactoring, but about how to avoid the need for refactoring. The XP guys should read Alexanders books. The idea of "vocabulary" in the macro rings as true to me as XP does in the micro.
For myself, I'm no longer uncomfortable at work. I realize that while I might not be saying "done" as often as others, I realize also that I'm not adding unseen or undetected, eventually certain, work to the system. Those who are responsible for figuring out if I'm doing good may, or may not, care about this. They may choose to place more importance on saying "done" alot than saying done and meaning it in a long term, years sort of way. I'm comfortable with that.
Fully agree. You have to climb very high in order to find paradise. Therefore, managing a team with disparate experience is close to impossible. The cost to train that team before is too high.
Boy, it seems that every time I turn around XP is getting slagged for one short-coming or another. Is the problem you described endemic to XP or the result of the way some teams are using it? There's quite a bit of variation in how teams approach the integrity of their systems. One thing that you didn't mention is XP's focus on design simplicity. The best teams I've worked do exactly what you were talking about. "Done" is defined by having found the right place for the code and put it there. They constantly assess how their variations fit into the big picture.
This morning I -finally- finished the book I've been working on. It is about working with large legacy systems, getting them under test, and making changes in them safely. I've worked with quite a few teams that have started to adopt XP and used it to help get out of legacy binds. Thinking back on it, understanding has been, as you said, the key thing. It seems that the big question for your team is why other people aren't doing more of what you do. You mentioned schedule pressure, but it seems that you're dealing with it in a different way then your team mates.
... > wanted, but not exactly. So, without a thought, he snarfed > up the code, put it close to where we were and made it do > what we wanted. Voila! Done! And I realized a couple of > things. First, I now knew why I was slower in "completing" > things than it seemed others were. Second, this is why XP > doesn't scale.
Same thing that Mike and essentially what you said. You weren't done and this wasn't XP. This was let's just get it to work and move on.
> It has been my contention for some time that XP doesn't > scale well in terms of organization size. That is, XP > works very, very well when the people working on the > system have essentially total system knowledge. However > there comes a point in the life of non-trivial systems > where it is impossible for any one human to understand the > system in the nuanced, detailed way that one must in order > to not end up create a pile of spaghetti.
You seem to be saying the same thing that Brooks says in Mythical Man Month and then later recanted when he understood the power of encapsulation and information hiding. Granted there should be an overall understanding of where things go, how the architecture works, etc. but knowledge of every nuance and detail of non-trivial systems is unnecessary and is in any case impractical if not impossible.
... > add it. To ABC. Not to XYZ. This avoids something that I > think is far, far worse than duplicated code, something XP > says is very bad. It avoids duplicated ideas.
XP doesn't really say anything... but I'd expect XP practitioners to also say that duplicated ideas are bad. Any time you need to change something and you have to change it more than once, that's bad.
... > and their attendant organizations. How does one ensure > that ideas are not duplicated in the system? I reject the > idea, entirely, that when one finds these duplications you > refactor them out with vigor. It sounds like a great idea, > but it ignores the real world of deadlines, making money, > and the expectations of business people. The fact is, and > I can say this from real world experience, when you find > these sorts of duplications you almost never have time to > remove them entirely. Other commitments have been made, > your time has been allocated in some other fashion, you > have a set of user visible functionality that has been > promised. And you must deliver in a bounded time frame. > The number of times the business can tolerate you saying, > "Right, found this duplication and I have to remove it. I > need to add 20% to my estimate" is substantially smaller > than the number of times you need to say it.
So what do you do when you find these? I tend to do what I can, make a note, and then wait for an opportunity to take another whack. After a while, there's a point when you just bite the bullet and finish it. It's hard if you're the only one doing this. It's a lot easier if the entire team is doing this.
... > context of the longevity of the system. It has to talk > about not just refactoring, but about how to avoid the > need for refactoring. The XP guys should read Alexanders > books. The idea of "vocabulary" in the macro rings as true > to me as XP does in the micro.
Perhaps System Metaphor or Domain-driven design is what you're looking for? I'm about 2 months behind on mail so I'm not sure if it's been resolved yet but there was a discussion about whether or not refactoring was "dark matter" that could/should be removed with more up-front analysis on David Anderson's Agile Mangement mailing list.
Re: XP guys should read Alexander's books This is a bit amusing... You do know c2 is called the Portland Pattern Repository right... :)
> For myself, I'm no longer uncomfortable at work. I realize > that while I might not be saying "done" as often as > others, I realize also that I'm not adding unseen or > undetected, eventually certain, work to the system. Those > who are responsible for figuring out if I'm doing good > may, or may not, care about this. They may choose to place > more importance on saying "done" alot than saying done and > meaning it in a long term, years sort of way. I'm > comfortable with that.
Now this sounds like you're just going to sit back and take it... and that's definitely not XP. Valuing Courage and all that. You can change this environment for the better.
> ... > > wanted, but not exactly. So, without a thought, he snarfed > > up the code, put it close to where we were and made it do > > what we wanted. Voila! Done! And I realized a couple of > > things. First, I now knew why I was slower in "completing" > > things than it seemed others were. Second, this is why XP > > doesn't scale. > > Same thing that Mike and essentially what you said. You > weren't done and this wasn't XP. This was let's just get > it to work and move on. >
As I understand it, XP practice would have been to continue after the code you two were working on passed its tests. Refactor the thing that varied for your code into new code, a method or class or however, and fold that chage back into the original, making sure that after the refactoring all tests continued to pass. You do have tests for all your code, yes? Then your code can call the original with the appropriate parameters. If there were no tests to act as a safety net for the changes, then one of the XP practices was not followed rigorously some time in the past. Michael Feathers is more or less correct in pointing out how XP is being inappropriately blamed.
> > there comes a point in the life of non-trivial systems
> > where it is impossible for any one human to understand the > > system in the nuanced, detailed way that one must in order > > to not end up create a pile of spaghetti. > > You seem to be saying the same thing that Brooks says in > Mythical Man Month and then later recanted when he > understood the power of encapsulation and information > hiding.
A side question for Jason: I've read MMM but I'm not familiar with Brooks' follow-up. I'd love to have a reference if you can get it.
I thought the purpose of pair programming was to some extent to prevent poor programming practices, like the copy-paste-tweak scenario you described. Why didn't you pipe up at the time with some gentle, but pointed chiding? I would have. I am not a proponent of pair programming at all, but it seems to me that you missed an opportunity to help this "incredibly bright kid" learn a good programming habit (or unlearn a bad one, as it were).
The argument about duplication and not getting time for refactoring seems a little fascile, because it needs to be contrasted with spending the time up front to avoid the duplication that necessitates the refactoring.
Who's Alexander? I just finished the John Adams biography a little while ago and I'm sticking on Alexander Hamilton, but I don't think that's who you're referring to...
One of the XP practices is Simple Design. Copy-paste-tweak does not lead to simple design. As I read it, you claim most everyone else on your team is doing this, which means it's bad design. This doesn't sound like an XP problem.
Perhaps your best potential argument in my understanding is that Simple Design (without duplication) is very difficult in in a large codebase and perhaps XP could do more to address how to do this. But in your situation, it sounds like the more immediate failure is that no one except for yourself is even trying.
Which, to me, brings us to the real problem: any process can be trumped by people.
My copy of Mythical Man Month, was only just recently released from Australian customs after I was finally able to get my shipping invoice to prove that my books were not new... so this is all of from memory...
Rick wrote: > where it is impossible for any one human to understand > the system in the nuanced, detailed way that one must in > order to not end up create a pile of spaghetti.
So as far as I remember Brooks says the same thing. I think he was referencing someone else... In any case, the gist is that in order for a system to not end up "a pile of spaghetti", at least one person (i.e., Chief Programmer/Architect) must understand the entire system in a nuanced, detailed way.
I wrote: > You seem to be saying the same thing that Brooks says > in Mythical Man Month and then later recanted when he > understood the power of encapsulation and information > hiding.
At around the time the first edition of MMM was being written Dijkstra had come up with this principle of information hiding. So Brooks comments that he doesn't think this will work.
In the 20th Anniversay edition, Brooks acknowledges that he was wrong. If you want to effectively build large systems, you need to exploit encapsulation and information hiding because it is impossible for any one to keep all the details in their head.
Steven wrote: > A side question for Jason: I've read MMM but I'm not > familiar with Brooks' follow-up. I'd love to have a > reference if you can get it.
It's in the new chapters in the 20th Anniversary Edition
> Who's Alexander? I just finished the John Adams > biography a little while ago and I'm sticking on Alexander > Hamilton, but I don't think that's who you're referring > to...
Found it. In the extra chapters Brooks says outright, "Parnas was right, I was wrong", regarding his assertion in Chapter 7, that encapsulation was a recipe for disaster.
> I have been, more or less, uncomfortable at work since > nearly my first day. We use an agile process and this > means, among other things, there is a brief daily meeting > where everyone presents what they did in the last 24 hours > and what they plan to accomplish in the next 24.... > > ... > That is, I spend real time understanding where something > should go, why it should go there, and what the impact on > everyone else is. I think this is valuable and required. > It allows one to reason about the system. If the ABC thing > is responsible for, say, describing the structure of the > persistent store, then when one needs to understand some > aspect of the structure of the persistent store they look > at ABC. If it's not there, it's not in the system and you > add it. To ABC. Not to XYZ. This avoids something that I > think is far, far worse than duplicated code, something XP > says is very bad. It avoids duplicated ideas. >
The daily scrum meeting that your team already uses can be used to avoid duplicating ideas by people paying attention to what others will be doing the next 24 hours. It helps to have someone with a broad knowledge of the code-base, though. On one project we had the we did this. The team was quite small, and I had managed to keep a birds-eye understanding of the code-base by going somewhat slower than the rest of the team. When someone said "I am going to be working on XYZ", I would say "cool, I have done something similiar with ABC, should we pair so we can reuse that code more easily?"
Previous Topic
