Summary
Managers need to measure the productivity of their employees, including that of developers. But choosing the right metrics to measure developer productivity is fraught with peril, says Steve McConnell in an article evaluating commonly used developer productivity measures.
Advertisement
Anecdotal evidence suggests that there can be as much as a factor of 10 difference between the productivity of individual developers. Yet, development managers cannot easily measure such productivity differences, mainly because commonly used metrics are seriously flawed, argues Steve McConnel, author of the classic Code Complete.
On real projects it’s hard to find a use for individual productivity measures that is both useful and statistically valid. In my experience, aside from research settings the attempt to measure individual performance arises most often from a desire to do something with the measurements that isn’t statistically valid. So while I see the value of measuring individual performance in research settings, I think it’s difficult to find cases in which the effort is justified on real projects...
Difficulties like these have led many people to conclude that measuring individual productivity is so fraught with problems that no one should even try. I think it is possible to measure individual productivity meaningfully, as long as you keep several key factors in mind...
One commonly employed productivity metric is the lines of code produced in a given time period or, in a variation on that theme, the number of function points produced in some time period. Yet, more code does not translate into more productive developers—just as writing less code to express a given problem doesn't automatically classify one as productive, says McConnell:
My observation is that there’s a correlation there [that great programmers can produce less code], but I wouldn’t make that statement that strongly. I would say that great programmers always write clear code, and that often translates to less code...
Sometimes the clearest, simplest, and most obvious design takes a little more code than a design that’s more "clever"--in those cases I think the great programmer will write more code to avoid an overly clever design solution. Regardless, the idea that productivity can be measured cleanly as "lines of code per staff month" is subject to problems either way...
McConnell suggests that, instead of measuring productivity along a single dimension, it's more meaningful to measure several dimensions, and even the trends along those dimensions. In addition, writes McConnell, it is also crucial to determine the true purpose of the measuring:
Don’t expect any single dimensional measure of productivity to give you a very good picture of individual productivity. Think about all the statistics that are collected in sports. We can’t even use a single measure to determine how good a hitter in baseball is. We consider batting average, home runs, runs batted in, on-base percentage, and other factors--and then we still argue about what the numbers mean.
If we can’t measure the "good hitter" using a simple measure, why would we expect we could measure something as complex as individual productivity using a simple measure? What we need to do instead is use a combination of measures, which collectively will give us insights into individual productivities...
Remember that trends are usually more important than single-point measures. Measures of individual productivity tend to be far less useful in comparing one individual to another than they are in seeing how one individual is progressing over time...
Ask why you need to measure individual productivity at all...In a real project environment, what do you want to use the measure(s) for? Performance reviews? Not a good idea... Task assignments? Most managers I talk with say they *know* who their star contributors are without measuring, and I believe them. Estimation? No, the variations caused by different design approaches, different task difficulty, and related factors make that an ineffective way to build up project estimates.
Some readers of McConnell's article pointed out that quality measures should also be included in the productivity metrics. Mark Roddy, for example, comments that:
Any discussion of productivity needs to take into account the level of quality of code produced versus a desired level of quality based on the needs of the project at hand. But then compared to productivity, quality is probably a much harder metric to measure.
This one is simple. No one will ever measure developer productivity. Period.
The best you can hope for it to get a sense of the developer productivity. To get a sense of it, it really helps to love the development process with all your heart (which is not true of the majority of managers), and to be a great programmer yourself (again not true of the majority of managers).
If you want to see the future, just look at writers. How does anyone measure a writer's productity? Do I hear you say "word count?" LOL!!! Hahahaha... Imagine if a paper like NY Times hired and fired journalists based on the word count? And this gets even funnier when you consider that in programming the goal is to perform a task with as FEW lines of code as possible, so very often, huge LOCs are an indication of incompetence rather than productivity.
How do you measure the productivity of research? When Thomas Edison tried one light bulb after another and failed over and over, was he being productive? According to Thomas Edison himself, YES! And I agree. Discovering failures is productive. But how will you ever convince your stereotypical pointy-haired-boss of that?
There are some smart managers out there who understand all of these things. But if the manager is on the wrong track, I imagine it will be very challenging to re-educate them, because they'd have to unlearn so much of what they believe about productivity, it might create a system shock.
Measuring programmer productivity is like measuring a painter's productivity, or an industrial designer's. We are not factory workers, we are designers. Every line of code requires thinking.
> Measuring programmer productivity is like measuring a > painter's productivity, or an industrial designer's. We > are not factory workers, we are designers. Every line of > code requires thinking.
One of the problems measuring productivity is that productivity, while important from an individual's point of view, is not nearly as important as efficiency. It's possible for someone to be highly efficient, while also being rather unproductive at the same time. (One example is when you're overwhelmed with various unrelated tasks - while you may be able to accomplish each with great efficiency, you may still be unproductive if, for instance, no meaningful outcome results from those actions.) So I think it'd be more important to focus on efficiency of a developer, or of an organization.
> One of the problems measuring productivity is that > productivity, while important from an individual's point > of view, is not nearly as important as efficiency. It's > possible for someone to be highly efficient, while also > being rather unproductive at the same time. (One example > is when you're overwhelmed with various unrelated tasks - > while you may be able to accomplish each with great > efficiency, you may still be unproductive if, for > instance, no meaningful outcome results from those > actions.) So I think it'd be more important to focus on > efficiency of a developer, or of an organization.
I take a complete opposite view. Just because you cannot measure productivity does not mean you shouldn't focus on it. Just because you can measure efficiency, doesn't mean you should focus on it.
People are addicted to measuring things and forget what life is all about. Life is not about cranking out lots of crap as measured by metric tons. Life is all about doing what you love. Money comes to those who understand this best.
The reason people pay for movies is because they love movies. The reason movies are good, is because people love making them. Just imagine how good would movies be if people loathed the process. The same goes for more pragmatic things like shoes. We love when our feet don't hurt. We love when the shoes are light. We love when they last through a beating. Now, a person who loves making shoes can make just the kind of shoe we all love wearing. And thus this kind of person can make a lot of money in the process. All commerce is strictly psychological. Even survival is a purely psychological value. It's not physical at all. There is nothing in particular in our bodies that either needs or wants to survive. Neurons don't care about what you think. Your skin doesn't care about how you feel. If you don't believe me -- investigate this matter yourselves.
Why do people love languages like Python and Ruby despite them having many flaws and being worse than LISP in many ways? It's because you can tell that Guido and Matz love what they do and you can feel it in their language. When you use it, you feel great. Programming is fun. When programming is fun, you can focus on satisfying other people's desires, because doing so is fun. So it's fun and joy all around. And this is where productivity comes in. You cannot measure it, but there is a difference between low and high productivity. Just because you cannot slap a number on it doesn't mean we shouldn't strive for it.
We should also recognize that productivity is as much, if not more, qualitative as it is quantitative. Someone who makes 10 pairs of shoes that each fall apart in 2 weeks is less productive than someone who makes 1 pair of shoes that lasts 30 years. Quality is key. Fun is key.
Forget efficiency. Efficiency is for robots. We are sentient beings. Let's enjoy life. Let's keep programming fun.
> Measuring programmer productivity is like measuring a > painter's productivity, or an industrial designer's. We > are not factory workers, we are designers. Every line of > code requires thinking.
Excellent point. The SE bunch stubbornly persists with the idea of "construction" of software and then, inevitably, with it comes the productivity concern. There is no such human activity as "construction" in software engineering. It's all design. The immediate result of programming activity (source code) is not the product itself, but a specification how to produce the product. If we can talk about construction at all, then the closest thing to it is the build process (compile/link), which is done by machines. And even that is just a metaphor because software is not a physical entity like bridge or car. Software is information.
> > Measuring programmer productivity is like measuring a > > painter's productivity, or an industrial designer's. We > > are not factory workers, we are designers. Every line > of > > code requires thinking. > > Excellent point.
Maybe it is, but that's not to say that there aren't productive programmers or unproductive programmers. It is still very valid to ask both "Which programmers are productive?" and "Just how productive are these programmers?".*
The knee-jerk reaction, that you can't measure programmers' output, implies that you can't differentiate between programmers. Clearly you can and, in a commercial environment, clearly you want to. Steve is right to ask how it can best be done.
> Excellent point. The SE bunch stubbornly persists with the > idea of "construction" of software and then, inevitably, > with it comes the productivity concern. There is no such > human activity as "construction" in software engineering.
Have you ever hired a home improvement contractor?
I'm not sure where the idea that construction, on a large or small scale, is a magically predictable activity orginated.
Though this may not happen for a long time, I believe that programmer productivity will eventually be measured by the reuse of a programmer's work-products. When one thinks about reuse it is often in the context of moving from one product to another. But, reuse also happens when a product is revamped, goes to the next version, or even when it goes to the next generation. Code that is ugly, cryptic, poorly commented, poorly structured, littered with bad programming practices, or otherwise a burden to a maintenance programmer is replaced, rewritten, and quickly forgotten. That work-product saw no reuse.
On the other hand, code and frameworks that withstand the test of time, provide long-term value, are well-commented, easy (or at least easier) to maintain, and supports the evolution of the product are carried forward from revision-to-revision, and perhaps even generation-to-generation. Programmer productivity in this case relates to the benefit the programmer provided to the original product plus the time and money saved with accompanying productivity gains realized by the later programmers in their efforts.
To quote more than one software visionary, attributing lines of code to productivity is grounds for professional malpractice. And, until more projects are developed with crisp requirements, function points, feature points, or other abstractions that could directly measure the contribution on a specific version of a product, assessing programmer productivity in the short-term will continue to be problematic. But, qualitatively, if not quantitatively, measuring the continue use or "reuse" of work-product (code, designs, architectural schemes, file formats, database designs, or even test harnesses and fixtures) can provide insight into who is doing quality work with long-lasting value, and who is doing hack-it-until-we-ship-it-worry-about-the-subtleties-later sub-standard workmanship that ultimately shifts costs from development to maintenance, service, and support.
Software management already has many of the tools at their disposal to know which of the work-products see this type of reuse, which do not, and who is responsible for each of the work-products in question. Whether most managers have the courage to act upon this data is different discussion altogether.
I agree entirely. I think it's possible both to assign absolute and relative values to productivity.
Relative productivity is usually easier to measure. If a certain number of people are doing basically the same type of work, then the more productive people are the ones who turn out more units of acceptable work in a given timeframe. Note that "unacceptable" includes both shoddy/erroneous work and work which is correct but too fancy.
Organizationally and individually you also develop a sense after a while of how long many tasks should take. In other words, a new task is similar in many or most respects to ones that you have experience with. In this manner you can gauge the productivity of a programmer assigned the task, based also on the expected skill level of the programmer. This could be viewed as measuring either relative or absolute productivity, depending on how you interpret it.
As for true absolute productivity, I think it can be measured, even if imperfectly at present. The intention of function point counting, for example, is intuitively right, even if the implementation can be criticized - how much business functionality is being delivered to a user by a given piece of software? If the requirements demand that an application do certain things, and both A and B meet the requirements, is it not a fair statement that if B took 2 times as long to finish that the team who created B is half as productive as team A?
I've just had a quick look at that article and, to be honest, it just looks like junk theory to me.
"Productivity is the rate at which you add further system tests (that pass) to the existing system."
Maybe I'm being naive here but tests are not part of the deliverable. To my way of thinking, productivity is a measure of useful delivery and is rather more complex than counting Lines Of Code.
(Dodgy analogy follows:) Tests are a tool used by developers in the same way that nails are used by a carpenter but you don't measure a carpenter's productivity by counting the number of nails they use to join two bits of wood, even if more nails make for a stronger join.
The first post responding to the article mentioned "...just look at writers. How does anyone measure a writer's productity? Do I hear you say "word count?"".
Watts Humphrey, who created the Personal Software Process, would track the number of pages he could create per day. Taking the average and accounting for distractions he was able to calculate his performance in writing books, which he was able to pass on to his publisher. With eerily accurate estimates he was able to make his deadlines and knew his own productivity, which other could count on.
If the PSP can work for writers, then it can work for developers using the PSP or Team Software Process.
Previous Topic
