Summary
Any software metric can be abused or misused, but does this mean we should avoid any and all software metrics? Joel Spolsky, Eric Sink, and I have an interesting, late night, conversation.

Yesterday (October 29th) I gave a talk on the CRAP metric and Crap4j at The Business of Software conference. Eric Sink and Joel Spolsky were also speaking at the conference.

At the end of the evening, I went to dinner with some of the other speakers and had an opportunity to meet and spend some quality time talking about CRAP and other software metrics with Joel and Eric.

Joel is not a big fan of software metrics in general. He is concerned that developers might end up writing code and allocating their time to satisfy a specific metric rather than writing the best possible code and allocating the time based on more important criteria. He narrated a couple of stories about horrific metrics misuse that he witnessed first-hand and was concerned that - in the wrong hands - the CRAP metric could be used in, say, performance reviews: You code is too crappy. You're fired!

I understand that there is potential, as well as some evidence, for software metrics misuse; but I don't think that's sufficient reason for avoiding metrics altogether. My reply to Joel was that if an organization/manager is so lazy and stupid to rely exclusively on any given code metric in evaluating programmers, then those programmers are probably better off being fired from that organization anyway. Better yet, the programmers would have great evidence to have the moronic manager fired.

While I understand that any tool, technology, or information can be abused by evil people and misused by stupid ones, I don't think we should use "How could this be abused or misused?" as the primary criteria - at least not without first balancing it against the potential benefits.

Toward the end of the conversation, Eric Sink observed that the argument was starting to sound a lot like the perennial "Guns don't kill people, people kill people" discussion. Great observation Eric.

What do you all think about software metrics?

Aren't you a bit surprised that, despite the fact that software runs the world and that we spend hundreds of billions a year writing and maintaining software, there isn't a single industry-wide metric that's being used with any consistency. Why is that?

Are all software metrics inherently evil and useless?

Is programming so much more art than science or engineering that it's pointless to try to quantify or evaluate code using objective criteria?

What do you think? Do you have any software metric horror/success stories to share?

Alberto

Talk Back!

Have an opinion? Readers have already posted 50 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Alberto Savoia adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Alberto Savoia is founder and CTO at Agitar Software, and he has been life-long agitator and innovator in the area of software development and testing tools and technology. Alberto's software products have won a number of awards including: the JavaOne's Duke Award, Software Development Magazine's Productivity Award, Java Developer Journal's World Class Award, and Java World Editor's Choice Award. His current mission is to make developer unit testing a broadly adopted and standar industry practice rather than a rare exception. Before Agitar, Alberto worked at Google as the engineering executive in charge of the highly successful and profitable ads group. In October 1998, he cofounded and became CTO of Velogic/Keynote (NASD:KEYN), the pioneer and leading innovator in Internet performance and scalability testing. Prior to Velogic, Alberto had 13-year career at Sun Microsystems where his most recent positions were Founder and General Manager of the SunTest business unit, and Director of Software Technology Research at Sun Microsystems Laboratories.