Summary
I see a lot of agreement in the software community poor quality code and other forms of technical debt slow you down. Where I see disagreement is on how to manage technical debt. What is your approach?
Advertisement
Gregor Hohpe, coauthor of Enterprise Integration Patterns (Addison-Wesley, 2004), asked "Where did all the beautiful code go?" in his keynote at TheServerSide Symposium last week. He made a plea for code quality, as described in a searchwebservices.com article:
As Hohpe sees it, code gets messed up for far more mundane and preventable reasons. Principal among them is that sloppy code begets sloppier code. His theory is that if the initial developer of an application doesn't take care to make his code clear and logical to any programmer that reads it, Pandora's box pops open.
The sequence of events Hohpe outlined begins with the developer writing code that is serviceable for the application – it runs okay – but the code itself is idiosyncratic and hard for another programmer to understand. The along comes a programmer doing maintenance a year after the application goes live. The code appears to be a convoluted mishmash.
So perhaps the second string maintenance programmer working on a revision, says to himself, "I'll just stick my code in here. How much worse can it get?"
After a few revision cycles like that an application that is getting a little long in the tooth may feature code originally developed by the best and the brightest, yet it's going from bad to worse.
Gregor concluded with:
"Write code that people aren't compelled to mess up," he said. "Make code so the next person sees you put thought into an elegant solution."
If you walk into a project that's in shambles—with bugs all over, a build that doesn't quite work—you're not going to have incentive to do your best work. But, if you go onto a project where everything is pristine, do you want to be the first one to make a bug?
The week before last at the Software Developer (SD) West conference, I saw part of a keynote by Uncle Bob Martin, in which he made a similar plea for code quality. His keynote was quite inspiring, and although his main message was that the prime directive of agile development is never be blocked, he touched several times on code quality. He reiterated his belief that, as he posted in this forum discussion a year ago:
The only way to go really fast is to write the best code you can possibly write, with the best design you can possibly give it. Any other technique will slow you down.
His position is that you should always avoid taking on any technical debt. At his SD keynote, Uncle Bob suggested that if you hack something up quickly to get it out the door today, that mediocre code doesn't just slow you down tomorrow, it slows you down today. This contrasts with Luke Hohmann's notion that to minimize risk just before a release, it is fine to take on some technical debt, but that you should clean it up just after the release, a technique he calls post-release entropy reduction.
Frank Sommers and I recently spent two months doing post-release entropy reduction at Artima, primarily by writing some code generators that generated much more consistent correct code than we could do by hand. It was quite expensive, and only made sense to me in the context of where we are planning to go. The more complexity you are planning to take on, the more important I think it is to invest early in building an architecture that will enable you to manage that complexity. But even though in the long term, building architectural pieces like test harnesses, code generators, and proper build scripts, pay off, in the short term they just cost.
I see a lot of agreement in the software community poor quality code slows you down. Where I see disagreement is on how to manage technical debt. What is your approach? When do you take on technical debt? When do you invest time to pay it off?
Several code quality tools were mentioned, and I would observe that, just as developer testing helps ensure correct behavior, code quality tools (style checkers, javadoc-coverage checkers, and more specialized checkers like PMD's cut-and-paste detector) are the things that will help to ensure "beautiful code." Although a few programmers -- those who understand and care about the concept of technical debt -- will do it by themselves, the rest need feedback to make them care about it. Without feedback, all it takes is one programmer who doesn't care to begin accumulating technical debt.
For example, we heard one story last week of a programmer who would regularly break the project build (most people have heard lots of stories like this). And then there's the "it works for me" maxim so often employed by students. So unless you can populate your team with programmers who can see the cost of technical debt without a reminder, I think some sort of gentle wrist slapper is necessary, just as it is with other aspects of programming.
It's strange. I was thinking about this just yesterday. I don't know if my comments will go down well, but I believe it to be an extension of what happens in the business world. It's all about building faster and cheaper. Better is not required as can be seen by the quality of software from major companies.
So putting things together faster and easier is given priority over quality. This has drawbacks. Anyone who remembers the big rooms that mainframes were in can attest to how "easy" it is to install. You bring in the machines and throw cables underneath. The guys installing it had a standard procedure for common hardware. But in the 80's (or whenever that was) when they were falling out of favour or getting smaller, old equipment was moved out and replaced. The thing was that it was not practical to pull out the old wires from the subfloor. You left them there unused. The new equipment would simply use newly installed cables. Just think of the mess of cables behind your computer if you have several devices or boxes.
What does this have to do with programming? Inserting code is usually much easier than removing old code because of the existing tangled mess of interactions that aren't clearly defined. Upkeep consists of three different actions 1) adding functionality, 2) replacing functionality and 3) removing functionality. Item 2 consists of both #1 and #3. Just by looking at a class definition, can you tell what functions it calls throughout the system and what it affects? Not unless it's a small class or application.
Now matter how good your foundation is, if you don't pay attention to the interactions, then you're doomed. And most languages don't pay attention to it because you don't need to during the building process. If you think about the interactions that will be possible in the future, it will slow you down. It requires you to think beyond the immediate necessary task. So languages don't put any emphasis on it and most languages outright make it convoluted such as in C++ and Java or worse Perl. Just to prove the point, is there a way to highlight everything that a certain class touches in any fashion? No, because in current languages, everything affects everything else because of implicit sequencing of execution. Executing a call before another can be quite different from reversing the calls. So there are effects that can't even be quantified in any concrete fashion.
You asked how to tackle it. There are no languages out there that emphasises interactions other than maybe Design by Contract. Other than that, we mostly deal with data (or objects) an not the interactions because they are too far reaching. It's too bad because if language designers would realise that data in motion is actually code, then we'd have much better programming languages. The concept of a function call is flawed. Unfortunately, it's what most languages are based on for interactions. If you were allowed to set up networks of interaction instead (without requiring execution to return to the caller), the situation would change.
The way to fix it is to remove the restriction of the return to sender concept of function calls. You can still have return to sender, but is no longer an obligation (and multiple execution paths can be implicitely be created). Nothing in the real world works in the return to sender paradigm. Yes, we can obtain confirmation of delivery. But normally, we can do other things in the meantime. Imagine if traffic (with cars) worked where you requested each car to move, but only one car could move at a time? It's ridiculous. It'd be like those puzzles with the one missing piece and you try to slide them around to get the correct result. This is the way we currently program! There are no clear definition of the interactions between the pieces themselves. Sometimes they interact directly with each other. Sometimes not. But remove just one piece and the whole thing starts breaking down.
Return to sender = BAD! Asynchronous networks = GOOD!
It's impossible to write code in current languages that someone else can't come around and mess up unless they have a clear view of the network of interactions. It's as simple as that. Don't even try to resolve the issue in contemporary languages because that network is not directly visible. If you can't see it, how can you fix it or modify it? Apologies for the long post, but it's difficult to explain the resolution without the background.
Is it just me or is the current debacle in Redmond over the new version of Windows a perfect example of technical debt that has overwhelmed a project.
I think it's pretty well established that the Microsoft method has been to shoehorn new features in and get the product out of the door as soon as possible. I've read and interview with a former MS project leader that not only admitted as much but espoused it as an effective method that MS has put to use. Specifically mentioned was the trouncing of Netscape.
I think the issue comes down to the understanding of what "cheaper" means. As long as it appears cheaper to just bash new code in without worrying about the longer-term effect on the project, then that's what will happen. And maybe in some cases this will actually be cheaper.
The problem is that by the time you discover that it wasn't cheaper to do it the sloppy way, it's too late to go back and do it right without it costing a lot more, so "doing it right" seems that much more expensive.
So the bigger problem is that it requires longer-term thinking in order to maintain code quality. If thinking is always in the short term, then you cannot make the case for longer-term practices.
What's worse is that there are situations where bashing out simple, messy applications is a reasonable approach. Witness the massive success of Visual Basic. If someone comes from such a background, it's even harder to convince them of the value of longer-term thinking.
> Several code quality tools were mentioned, and I would > observe that, just as developer testing helps ensure > correct behavior, code quality tools (style checkers, > javadoc-coverage checkers, and more specialized checkers > like PMD's cut-and-paste detector) are the things that > will help to ensure "beautiful code." Although a few > programmers -- those who understand and care about the > concept of technical debt -- will do it by themselves, the > rest need feedback to make them care about it. Without > feedback, all it takes is one programmer who doesn't care > to begin accumulating technical debt. > I agree with that. In the past few months we integrated into our build several static analysis tools, and we plan to integrate more. We have checkstyle to enforce our coding convention (which is really Sun's coding convention). We have FindBugs integrated, and plan to add other tools like PMD and JLint when we've finished cleaning up everything that FindBugs finds.
Also, part of our refactoring was to rearrange the package structure so we have layers, and I wanted a tool that would check that we always adhere to layering rules (like only com.artima.persistence is allowed to be coupled with Hibernate). There's something called Macker that I was planning on taking a look at, but it would also be quite easy to write a quick Ruby script to check the layer rules.
I also wanted a visualization tool that gave me insight into dependencies. I tried JDepend, but I really couldn't figure out how to make much use of it. I am currently looking at Dependency Finder and ClassCycle. Lastly, we integrated Emma in our build to help us visualize our test code coverage.
There's a nice list of open source static analysis tools for Java here:
Part of architecture in my mind is building development infrastructure that makes it easy to do things the right way, and uncomfortable to do them the wrong way. Java has a lot of free and commercial static analysis tools that can form part of that infrastructure.
> I think the issue comes down to the understanding of what > "cheaper" means. As long as it appears cheaper to just > bash new code in without worrying about the longer-term > effect on the project, then that's what will happen. And > maybe in some cases this will actually be cheaper. > > The problem is that by the time you discover that it > wasn't cheaper to do it the sloppy way, it's too late to > go back and do it right without it costing a lot more, so > "doing it right" seems that much more expensive. > I think doing it right often is more expensive in the short term. That cost is an investment that may or may not pay off over time. That's why I don't agree with Uncle Bob's view that you should always write the best code you know how to write. I think you need to think like an investor when you manage software projects and write code.
> So the bigger problem is that it requires longer-term > thinking in order to maintain code quality. If thinking is > always in the short term, then you cannot make the case > for longer-term practices. > > What's worse is that there are situations where bashing > out simple, messy applications is a reasonable approach. > Witness the massive success of Visual Basic. If someone > comes from such a background, it's even harder to convince > them of the value of longer-term thinking.
I don't see it as worse that sometimes you find yourself in situations where compromising on code quality is the appropriate path. Some cases where I think it is reasonable is:
1. When you don't have any experience in the domain. You're probably going to do it wrong anyway, so get something out the door so you can get real-world experience in the domain. That's how you learn how really to design the system, and then you can apply that knowledge later by refactoring or rewriting the app.
2. When you need to meet a deadline because you promised something to customers by a certain date. You gain customer good will be keeping your promise, and can go back after the release and spend some time doing post-release entropy reduction to pay off the technical debt quickly.
3. When you think there's a market window you want to hit. In the second case, if you were right and are successful because you were first to market, you can try and go back and clean it up later. Sure, it will be more expensive to clean it up later, but if you were right about the market window, then in theory you will be able to afford it and it will be a net win for you economically.
4. When you're right before a release and you need to fix a bug. Even if you have high test coverage, I just don't think that's the time to do major refactors. Just do what you need to do to fix that bug with minimum risk, but go back and clean it up after the release.
It isn't wrong to think short term sometimes, in my opinion. You have to find the best balance between short and long term goals. I think the main problem is not that you take on debt sometimes to meet short term goals, but that you lose control of that debt over time. If you don't manage your code quality, the entropy will eventually overwhelm you.
I will have to say here that the concept of "we'll clean it up after release" brings only one word to mind. Bullshit. If you can't get it in in development then it will probably never get done. Even if the owner of the company tells you you can clean it up after release (as mine has on a few occasions) the time will never be found. There is always something more urgent, you can do it after that and there is always something important that will suddenly become urgent because it has been put off because of all the other urgent things. Do it when you can, even if it means lying about what you are doing.
Coding as guerrilla warfare is the only attitude that will get any results in the companies that I have worked for. Hell I have had people complain when I have reformatted the source code and removed commented out sections of code that I wasn't supposed to be working on. Anyhow...
If you don't do it now it will never get done no matter what someone says at a conference.
> If you were > allowed to set up networks of interaction instead (without > requiring execution to return to the caller), the > situation would change.
> The way to fix it is to remove the restriction of the > return to sender concept of function calls. You can still > have return to sender, but is no longer an obligation (and > multiple execution paths can be implicitely be created).
> There are no clear definition of the > interactions between the pieces themselves. Sometimes > they interact directly with each other. Sometimes not.
I had similar thoughts and I did some experiments using a network of interacting components (a virtual machine was responsible for coordinating synchronous and asynchronous interaction between components). I wouldn't go as far as to say that nothing works with standard function calls, but I can see some advantages of using the interacting components approach. In my experiments the components communicated through typed ports (port variables), not by invoking interface methods, so when you want to send data to another component you just specify the output port of the sending component and the VM would route it to an input port of the receiving component (based on the connection specification defined separately). I used 'connectors' construct that connects two components together and can do some additional processing before the data reaches the receiving component. Anyway, I think it's an interesting approach, but it certainly requires a lot more effort to find out its true benefits.
> I will have to say here that the concept of "we'll clean > it up after release" brings only one word to mind. > Bullshit. If you can't get it in in development then it > will probably never get done. Even if the owner of the > company tells you you can clean it up after release (as > mine has on a few occasions) the time will never be found. > There is always something more urgent, you can do it after > that and there is always something important that will > suddenly become urgent because it has been put off because > of all the other urgent things. Do it when you can, even > if it means lying about what you are doing. > > Coding as guerrilla warfare is the only attitude that will > get any results in the companies that I have worked for. > Hell I have had people complain when I have reformatted > the source code and removed commented out sections of code > that I wasn't supposed to be working on. Anyhow... > > If you don't do it now it will never get done no matter > what someone says at a conference.
Maybe that's partly where Uncle Bob is coming from when he says you should always write the best code you can. (He said that at a conference too, in an uplifting keynote at SD two weeks ago.) Although he didn't mention this problem of not having time to go back and do it again as a justification. He said without elaborating that if you cut corners today it will slow you down today.
I've had the same experience you describe in places I've worked. There just is no time to ever go back later, unless the code is such a mess that it becomes a crisis. I've seen that happen a few times over the years.
It is really up to management. Management has to manage the quality of the code, and make the bets on when to cut corners to meet short-term goals, when to slow down short term feature progress to meet code quality goals. They need to communicate it to developers and developers need to follow that lead.
I think it is important to invest in quality early in a project if you're planning on this project lasting a long time and becoming quite large. What you invest at the beginning, pays off handsomely if the project does last and grow, because the architecture (and culture) you put in place guides the development over time, steering it along a quality path. It's darned expensive though, and really hard to justify, especially when higher management doesn't really appreciate the long-term benefits of quality.
> Maybe that's partly where Uncle Bob is coming from when he > says you should always write the best code you can.
If you look at that statement, there's a lot of lattitude in saying "the best code you can." I've had a number of experiences where I create a tool "the best way I can," which is not for publication but to get a job done. I may not know what I'm doing in which case the code may be hacky. Then, as I use the tool more and understand its variabilities, I improve it -- mostly because I can see it has a longer lifetime than a one-off program, and I want it to be easier to modify. As time passes it improves each time I touch it.
Of course, I am management in this case, and I also consider readability one of the most important aspects of code.
> > Maybe that's partly where Uncle Bob is coming from when > he > > says you should always write the best code you can. > > If you look at that statement, there's a lot of lattitude > in saying "the best code you can." I've had a number of > experiences where I create a tool "the best way I can," > which is not for publication but to get a job done. I may > not know what I'm doing in which case the code may be > hacky. Then, as I use the tool more and understand its > variabilities, I improve it -- mostly because I can see it > has a longer lifetime than a one-off program, and I want > it to be easier to modify. As time passes it improves each > time I touch it. > > Of course, I am management in this case, and I also > consider readability one of the most important aspects of > code.
I got the feeling part of it was that Uncle Bob believes you should always use test-driven development, and that in itself would get you most of the way towards the best code you can possibly write. He's also I think using a bit of hyperbole to inspire people to write better code. He says you should have 100% unit test coverage, but in his keynote he did mention that you're probably not going to get all the way to 100%, but you can get close. 95% 98%.
> > I think the issue comes down to the understanding of > what > > "cheaper" means. As long as it appears cheaper to just > > bash new code in without worrying about the longer-term > > effect on the project, then that's what will happen. > And > > maybe in some cases this will actually be cheaper. > > > > The problem is that by the time you discover that it > > wasn't cheaper to do it the sloppy way, it's too late > to > > go back and do it right without it costing a lot more, > so > > "doing it right" seems that much more expensive. > > > I think doing it right often is more expensive in the > short term. That cost is an investment that may or may not > pay off over time. That's why I don't agree with Uncle > Bob's view that you should always write the best code you > know how to write. I think you need to think like an > investor when you manage software projects and write > code. > > > So the bigger problem is that it requires longer-term > > thinking in order to maintain code quality. If thinking > is > > always in the short term, then you cannot make the case > > for longer-term practices.
alas, the entirety of humanity works politically (lower case intentional). consider the New Orleans levees: it was never in the original politicians best interest to actually do it right. only to do something. thus, as cheap (esp. deviously, "I'll build the good citizens of New Orleans the best levees possible") as possible. they won't be around to pay the piper. same with anything more than the most horizontal software. anything a tad vertical is about lock-in. once the sale is made, and the client nicely locked-in (making the logic of the business process tied to the software, rather than vice-versa, is a good strategy), then hiring $1/day Elbonians charged at $100/hour is a money spinner. doing the software right the first time is really not in the VAR/ISP best interest. > 3. When you think there's a market window you want to hit. > In the second case, if you were right and are successful > because you were first to market, you can try and go back > and clean it up later.
that's never done. it's more profitable to squeeze. or more devious: charge the first client to discover a bug a fee to fix it.
> I think the main problem is not that > you take on debt sometimes to meet short term goals, but > that you lose control of that debt over time. If you don't > manage your code quality, the entropy will eventually > overwhelm you.
again, real world: the debt takers are Pointy Haired Managers; the debt payers are coders. in the world of economics capitalists are often described as simply entities out to socialize costs and privatize profits (the recent Wal-Mart poor mouth about health care comes to mind); the case of software is analogous.
Quality is context-dependent. Long ago I programmed video games for the Atari 2600. We did things like jump into the second byte of an instruction just to save a ROM byte. With a very limited platform (128 bytes of RAM, 4K ROM in cart, no interrupts, no image buffers, etc) tricks like this made the difference between making the game possible or giving up. The context for typical applications today is quite different, so a different approach to quality is appropriate.
Quality is subjective. We can all agree that bugs are bad, but what about naming conventions, commenting style, maintainability or extendibility? Great software has been created using all kinds of styles and conventions (even with mixed styles). Often goals are contradictory: adding extra abstraction may make the code easier to extend but may also create a greater learning-curve for more mundane maintenance. It's fine to say "do it right the first time", but the problem is there is no "right" way.
I'm not sure about the technical definition of "debt", but the fact is that all code constrains you in some way. You're not going to choose between debt and non-debt software. At best you get to choose how much debt and what type of debt you want to live with.
To a great extent your choices are a gamble on the future and more abstractions won't always win the day. For example, if Artima had started in the pre-Internet era as a BBS, it's not likely that the much of that code would be useful for the artima website no matter how well designed it was.
Flat View: This topic has 97 replies
on 7 pages
[
123456
|
»
]
Previous Topic
