Ignoring statements by some proponents of agile processes that suggest or state commenting is not required in well factored code, most folks I talk with place some value on comments in code and do it with varying degrees of religiousness. One of the real downsides of commenting code, and a point not lost on or by the extreme XP (XXP?) set, is that commentary in code has to be maintained. Why this is so becomes quickly and sometimes cruelly obvious if you, as a client of some chunk of code, make the assumption that the comments actually reflect reality and they don't. I liken this feeling to me playing basketball with an NBA star. I'm driving to the basket all set to score and this guy fakes left, fakes right, steals the ball and I'm left there with my finger in my nose muttering an eloquent "Huh?".
The issue of course is one of security premised on assumption. The assumption that the comments are up to date and accurately describes what the code is doing gives one a false sense of security when making use of the behavior described. It doesn't take too terribly long to realize that there are some comments you can trust, and some you probably shouldn't. The JDK contains documents you can probably trust, ignoring bugs and other unintentional deviations. Meanwhile it's at least been my experience that comments at ones own work should not be trusted (unless you work for Sun and you happen to work on the JDK of course). It's just not part of the product and vanishingly few development organizations are concerned with long term efficiency to the point of investing in accurate code commentary.
If you've been in the business for long enough, say about a month, the above observation should strike you as nothing if not banal. You're either nodding your head knowingly, resigned to the realities of professional programming or you're lucky and you've found a place where the fish always bite, metaphorically speaking. Best just to keep it quiet and keep reeling them in. Anyway, for the nodders in the group, I'm actually here to talk about unit tests.
Psych.
Ignoring several realities, writing code that implements well stated functional requirements is pretty straight forward. You take stuff in, apply some transformations, and generate a result. Unquestionably some things are harder than others, and I'm not going to spend time discussing or defining "well stated". Suffice to say that if someone can tell me what something is supposed to do I can build something that will do it. But, as I said, I'm ignoring several realities in making that statement. Some of those realities are that the thing has to finish in some reasonable period of time. The thing can't just crash or fail in the face of erroneous but reasonably anticipated input. In some environs these issues fall under the heading of non-functional requirements. Personally I just call this sort of stuff the fun bits. Anyway.
Unit tests are almost exclusively focused on what I'm calling the easy bits of a code chunk. Given a set of inputs does it generate the correct output sort of thing. They seem to rarely, if ever, pay attention too much to detecting race conditions, deadlocks or combinatoric performance issues. And this can, probably does, lead to a false sense of security. An assumption that the unit tests define the totality of a systems behavior. Unfortunately or fortunately, depending on your perspective, this isn't the case.
It's probably the case that writing unit tests that verify that there are no deadlocks in a system are simply too expensive to write with the routine frequency of functional unit tests. Or maybe it's too hard, one begetting the other. I dunno. But, at least where I've looked, there aren't too many of these non-functional unit tests. So you can't really take unit tests at face value. Probably, unless you're working on something small, or a single user app, you need to reserve time in the schedule to test for these things and, more, time to fix them. Unless your crew happens to be filled with nearly omniscient demi-gods the likelihood of a non-trivial system having performance issues and such approaches certainty.
So if you're looking at your unit test results and you're congratulating yourself on building a quality chunk of the world or, more likely, someone not quite so in touch with the system is doing the congratulating for you, be aware that it can pay large dividends to ask "Are you sure?"
One difference between comments and unit tests is that comments can lie by commission, but unit tests only lie by omission. Neither comments nor unit tests are a cure-all. Both require dilligence and discipline on the part of the programmers. A programmer who does not maintain his comments will probably not make sure that he has proper unit test coverage.
As an XPer, I have nothing against comments in general. My attitude towards them is that they are a necessary evil. I try not to use them if at all possible. I try to make the code express my intent to the greatest degree possible. I only use comments when I fail.
This is not a religious thing. It's just the application of a value. If you looked at my code you'd find a fair number of comments. This is because I'm not really very good at expressing my intent in code. On the other hand, I *try* not to comment. I try to make the code stand on it's own. I value the fact that code cannot lie.
I also try to make my unit tests communicate my intent. Well written unit tests form a kind of thread-of-consciousness description of my intention. If you read them from beginning to end you'll see how I was thinking about the program, and the order in which I thought those things. This style allows others to follow my thinking, and find gaps in it.
It is *possible* to write unit tests for race conditions; but coverage of all possible race conditions requires that we understand where they can occurr. In my experience most race conditions are a surprise. The developer simply didn't anticipate that the system would block or delay at a certain point, or that another thread could get to the place it got to.
The article misses an important distinction. Unit tests get checked by the machine every time they are executed, whereas comments are not. Only human diligence keeps comments correct. Since humans are not multi-tasking, thought is a precious resource. Why waste this resouce on something a machine can do? Computers are good at doing repetitive tasks and testing is one of the most repetitive tasks around. Any comment that can be expressed as a test should be.
As for multi-threading issues, I feel your pain. I've been writing unit tests for multi-threaded i/o code for a couple of years now and it's not easy. But it does get easier over time. I also *listen* to the code; if something is hard to test then perhaps it's a signal that the design is too complicated. For instance, limiting most thread interactions to simple producer-consumer relationships greatly reduces the risk of race conditions and deadlock.
>Rick Kitts: >It's probably the case that writing unit tests that > verify that there are no deadlocks in a system are simply > too expensive to write with the routine frequency of > functional unit tests
Unit tests only test the unit so they aren't really in a position to find deadlock conditions as they are normally caused by interactions.
With the emphasis on mocks testing has become even less useful at finding real bugs.
Anyway, deadlock is prevented largely by architectural choices, not testing, but that only works for those of us who still believe in architecture :-)
>Robert C. Martin: >I only use comments when I fail.
You don't know when you have failed. Only the reader knows when you have failed because only they know when they don't understand. Your selection of objects, names, tests etc are all from your POV. In reality the amount of information communicated by your choices is a combination of the reader, context, and your efforts. Vary those and like a rainbow you get different results.
What is boilded out of all the naming choices is the intent and the why. You can get rid of a lot of comments by good developement techniques, but the why must still be written down. Without the why you don't have a strong foundation for future evolutionary paths.
>Robert C. Martin: >I also try to make my unit tests communicate my intent
Unit tests are just more code about other code. Where does the symbol get grounded? At having looked a lot of unit tests seldom do i feel it enhances by understanding. Do you have any good examples you can point to?
>>Robert C. Martin: >I value the fact that code cannot lie.
Everything in code can lie. A name may not be representive of what it really is. A test may not really test for what it tests for. A method may do more than it says. A method may do less than it says. A method may do something wrongly. Issues may have not been tested for. Problems may not have been addressed. Requirements may not have been met.
>Ian Rae: >Unit tests get checked by the machine every time >they are executed.
While that offers some comfort, all you know is that tests pass or fail. You don't know anything about the quality of the tests or implementation. You can't trust comments, but you can't trust code either. The same people write both and their weaknesses are evident in both.
>You don't know anything about the >quality of the tests or implementation. You can't trust >comments, but you can't trust code either. The same >people write both and their weaknesses are evident >in both
Well that's true about anything. People are fallible. The issue is what should a programmer spend time on! Is a good comment worth more than a good unit test? I would argue that a unit test provides more value, especially in the long term, when you try your code on a new combination of OS + service pack + 3rd party library versions. Or your Arianne 5 rocket exceeds 32767 cm/sec horizontal velocity! I believe the Arianne 5 is a case of documented or commented behaviour (variable x must not exceed sizeof(int)) that no one read.
>Unit tests are almost exclusively focused on what I'm calling the easy bits of a code chunk.
The course I took on system testing (Sandy Sorkin http://www.coastaltech.com/) had some industry statistics, based on studies of huge codebases. One fact was that by executing a single execution path through a function you will find on average 50% of the bugs. Two execution paths will net 67%.
This is a huge payoff -- two unit tests per method and you catch on average 2/3 of the bugs. Leaving you time to focus on the hard bits.
Wearing a seat belt will only save you in the "easy crashes". This doesn't mean you wear one.
>Well that's true about anything. People are fallible. The >issue is what should a programmer spend time on!
The question for me is what should a programmer do to produce the best system for all audiences. It's not zero sum, all or nothing.
>Is a good comment worth more than a good unit test?
They don't have the same purpose so their worth isn't directly comparible in the way you want to compare them. What has more value, your liver or your heart?
>I would argue that a unit test provides more value, >especially in the long term, when you try your code on a >combination of OS + service pack + 3rd party library >versions. Or your Arianne 5 rocket exceeds 32767 cm/sec > horizontal velocity!
I am sure they had a zillion tests. Unit tests don't catch these sort of errors. Testing with mocks certainly do not find this sort of error. A comment wouldn't find this kind of error, but that's not what comments are for.
>I believe the Arianne 5 is a case of documented or > commented behaviour (variable x must not exceed > >sizeof(int)) that no one read.
A lot has been written on this, but as it was cooperating units that failed, how would mocked unit tests mean anything? The code did all it could do on something that was never supposed to happen. Their system tests simply didn't have enough coverage. I think if they had used typed objects instead of ints they could have found the problem at compile time, which is far better in critical systems than relying on testing.
>The question for me is what should a programmer do to produce the best system for all audiences. It's not zero sum, all or nothing.
Couldn't agree more. And for each programmer there will be a different best mix of design, coding, debugging, unit tests, code reviews, etc.
>They don't have the same purpose so their worth isn't directly comparible
I've not heard a clear definition of the purpose of comments. They tend to be a vague mish-mash of descriptions of intent, warnings about gotchas, to-do items, corporate coding standards, and so on. If you're saying that things better expressed as a unit test shouldn't be in comments then we agree.
The problem with comments or others forms of documentation is that they get out of date. They get out of date because they are not automated. Wherease unit tests make accurate statements about the current abilities of the system. These statements can of course not be the whole truth as you say. I think it was Djikstra who said that software confounds human common sense. A car engine can be pocked and grimy and keep on working. But a single bit wrong in a trillion bit program can render it completely useless. Programmers are "changing bits" all the time; so being able to say anything truthful about a piece of software is very valuable.
>A lot has been written on this, ... Their system tests simply didn't have enough coverage
That is a understatement. The rocket blew up in the first minute of flight! There were no large irregularities in the flying conditions. Whatever tests were done didn't even test the main code path.
AFAIK the Arianne did not have unit tests, and the problem was that the Arianne 4's code was reused on Arianne 5 without sufficient checking of the codes ability to handle the 5's higher velocities. Basic unit tests would have caught this. (The Arianne case is both a good one for unit tests because it demonstrates the cost of a simple bug, and a bad one because real-time embedded code happens to be one of the hardest types of software to unit test. I still use it because most of us are not writing real-time code).
Previous Topic
