Summary
The ICT security community is suspicious of agile processes. "They do not produce formal documentation" is an often-heard complaint. Agile developers, on the other hand, blithely ignore security concerns.
Advertisement
I led a discussion at the Belgian OWASP chapter meeting last week on whether agile processes are capable of producing secure software. Received wisdom in the security community has it that they cannot. Security professionals object to the lack of formal documentation. They argue that without formal specifications, there can be no credible assurance argument. The agile community, for their part, has blithely ignored security concerns.
I do not believe formality is the issue. Formal specifications are not a virtue in themselves. They are only useful to the extent that they aid communication. Often the demands security officers put on developers get in the way of communication, rather than furthering it. Developers find them overly bureaucratic and a hindrance to getting their job done.
To many present, agile development was new. Initially, the reaction was very cautious, especially from the security professionals present. The ice broke when one of them observed that while defending against attacks, the ICT security profession tends to work in a very agile way: do, observe, steer, do - iterate as quickly as possible.
So there is an agile mindset on the development side, an agile mindset on the security side, but what passes in between is distinctly non-agile. It does not have to be like that. We have to get rid of the wall between the two.
I don't know if it started with CMM, but it seems to me that a lot of the IT world considers documentation to be more important than results. The reason I mention CMM is that the above is in-line with level 2 CMM (which seems to be where most companies settle in) which, for anyone who does not know, is the level where you have documented your proceses. It says nothing about whether thess documents are actually correct and whether the process is a good one.
The formal documentation requirement is a religious idealogy. For example, where I work, we expend a large amount of time and money producing document after document for every step of the development process. I can't find a single developer or support person who finds these documents useful. Nevertheless, we are constantly hounded by management to produce these documents (more are being added) and more stringent controls are being put in place to ensure the proper doucments are submitted in triplicate and approved before any developer uses the restroom.
Documentation is good, no doubt and I used to be very suspicious of Agile methodologies. But after the experience I have had at this job, I am dying to try something Agile. Anything but this soul-crushing excercise in futility.
It is important to remember that requirements in Agile Methods are intensely formal. Every feature is documented as a suit of automated acceptance tests. The design of every module is specified as a suite of unit tests. These tests are written before the code that makes them pass. It is difficult to be more formal than that.
Indeed, Agile Methods are not about informality. Rather they are about just-in-time FORmality. It's not that we don't write the formal documents, we do; and they are more formal than any document ever written in a waterfall domain. The difference is that we don't write them all up front. We break the project up into very short iterations, and deliver just those automated acceptance tests that we need at the start of each iteration. We deliver the unit tests on a minute-by-minute basis.
I need to sit down and write this up at some point, but the idea has been rattling around my head for a while. George Lakoff explains the divide between liberals and conservatives through the family model frame: liberals correspond to a nurturant parent model and conservatives to a strict father model. I believe the same metaphor can do a lot to explain the split between agilistas and CMM'ers. And I think the fact that I haven't said who is which, but you know it anyway, suggest that it's a very effective metaphor.
> I do not believe formality is the issue. Formal > specifications are not a virtue in themselves. They are > only useful to the extent that they aid communication. > Often the demands security officers put on developers get > in the way of communication, rather than furthering it. > Developers find them overly bureaucratic and a hindrance > to getting their job done.
Perhaps there is an amibguity in the meaning of "formal specification" because formality can mean formal in a bureaucratical or in a mathematical sense. I think the latter is definitely a virtue because it enables software that can be proofen to be consistent and it can prevent severe bugs before a line of code is written.
> It is important to remember that requirements in Agile > Methods are intensely formal. Every feature is documented > as a suit of automated acceptance tests. The design of > every module is specified as a suite of unit tests. These > tests are written before the code that makes them pass. > It is difficult to be more formal than that.
This is firmly the view from the development camp and it embodies the problem specified. As far as the customer is concerned, 'automated acceptance tests' are code. In order to write such tests for a module, there must have been an earlier decision that a module was required that has a particular function.
This is the gray area where Agile methodology is weakest. Automated tests exist to ensure the code meets the customer's specification. They are not, in themselves, the customer's specification but the developer's interpretation of it.
> Indeed, Agile Methods are not about informality. Rather > they are about just-in-time FORmality. It's not that we > don't write the formal documents, we do; and they are more > formal than any document ever written in a waterfall > domain. The difference is that we don't write them all up > front. We break the project up into very short > iterations, and deliver just those automated acceptance > tests that we need at the start of each iteration. We > deliver the unit tests on a minute-by-minute basis.
Whether or not the tests and code can be written on a minute-by-minute basis is a moot point, they must both be delivered to meet some preceding specification. Without such a specification, what is the motivation for creating the test?
I have no idea which ones you have in mind. Are agile methods leftist because they seem to be egalitarian? Or are they rightist because they are so full of discipline. Am I being leftist when I wear my green "Test-First" wrist band as a committment that I WILL TEST MY CODE? Or is that more conservative? If you pair program are you also in favor of Gay Marriage? If you practice continuous integration does it mean you are pro life?
In short, you've managed to raise my ire with this statement. The fact that our nation is politically polarized should NOT spill over into our professional views. Agile Methods do not have any particular political bias. They are neither right nor left. They are SOFTWARE methods.
A year ago I was at an Agile conference in Europe. While drinking a beer with a gaggle of attendees I heard one of they say something to the effect: "Leftists care about people. Agile cares about people. Therefore Agile is leftist." I had a two letter response for him.
His premise was wrong on both counts. Left vs. Right is not a measure of how much you care about people. And caring about people is not a measure of Agility vs Waterfall.
> > It is important to remember that requirements in Agile > > Methods are intensely formal. Every feature is > documented > > as a suit of automated acceptance tests. The design of > > every module is specified as a suite of unit tests. > These > > tests are written before the code that makes them pass. > > It is difficult to be more formal than that. > > This is firmly the view from the development camp and it > embodies the problem specified. As far as the customer is > concerned, 'automated acceptance tests' are code. In > order to write such tests for a module, there must have > been an earlier decision that a module was required that > has a particular function.
Quite to the contrary, well designed automated acceptance tests read like formal requirements and bear little resemblance to code. (see www.fitnesse.org). They are readable by anyone technical enough to understand the features of the product.
What's more, these tests are written first, before the application is designed. They constrain the design of the application to be testable. The tests, once written, require certain forms of access that the design of the application must deliver.
I teach a course on FitNesse in which I hand a team of developers several dozen acceptance tests and instruct them to make those tests pass. When they are done, they have made a complete program. The tests arethe specification. > > This is the gray area where Agile methodology is weakest. > Automated tests exist to ensure the code meets the > e customer's specification. They are not, in themselves, > the customer's specification but the developer's > interpretation of it.
Again, to the contrary, the automated acceptance tests are written by business analysts and QA. They are not written by the programmers. Programmers write their own automated test suite in the form of unit tests. But the acceptance tests are written by the people whose job it is to specify and verify the requirements.
> Whether or not the tests and code can be written on a > minute-by-minute basis is a moot point, they must both be > delivered to meet some preceding specification. Without > such a specification, what is the motivation for creating > the test?
The preceding specification is necessarily less formal than the acceptance tests. Morevore the preceding specification may be preceding only by a few hours or days. Indeed, the preceding specification may never be represented by more than an index card with a word or phrase on it (i.e. a story card).
It can well be that the acceptance tests are the only written form in which the requirements are described in any detail. All other forms may be verbal or mnemonic. It is not uncommon for the acceptance tests to be sketched on a whiteboard by the BAs and QAs during the iteration planning meeting, as a way of telling the developers what they intend to test. Those completed ATs arrive a few days later.
> I don't know if it started with CMM, but it seems to me > that a lot of the IT world considers documentation to be > more important than results. The reason I mention CMM is > that the above is in-line with level 2 CMM (which seems to > be where most companies settle in) which, for anyone who > does not know, is the level where you have documented your > proceses. It says nothing about whether thess documents > are actually correct and whether the process is a good > one. > > The formal documentation requirement is a religious > idealogy. For example, where I work, we expend a large > amount of time and money producing document after document > for every step of the development process. I can't find a > single developer or support person who finds these > documents useful. Nevertheless, we are constantly > hounded by management to produce these documents (more are > being added) and more stringent controls are being put in > place to ensure the proper doucments are submitted in > triplicate and approved before any developer uses the > restroom. > > Documentation is good, no doubt and I used to be very > suspicious of Agile methodologies. But after the > experience I have had at this job, I am dying to try > something Agile. Anything but this soul-crushing > excercise in futility.
The documents you are talking about are formal in the sense that they have a rigid form but with no clear measure of substance. Agile methods produce documents that also have a rigid form, but that also have a strongly measured amount of substantive content. The primary requirement of the documents produced by an agile team is that they must execute. This does not mean that they are code. It means that they are so unambiguous as to be interpretable by a machine and reliably compared to the system being built.
These documents are produced incrementally, one iteration at a time, rather than all up front. They are produced minutes or hours before the code that they describe is written. Indeed, they are sometimes produced concurrently with the code.
So be careful what you wish for. A switch to agile is not a switch away from formal documents. You may, in fact, find that the formality and discipline increases with such a move. However, you will also find that you are writing code a lot sooner. ;-)
I often hold one-day Agile Overview seminars for company executives explaining how this works. The remarkable thing is that they immediately recognize it as they way in which they run virtually every other part of their business. Short cycles, lots of feedback, and measurable results are not foreign to business people. They've just been told over the years that you can't run software that way. When they hear that software can be run like every other part of the business, they get interested.
> These documents are produced incrementally, one iteration > at a time, rather than all up front. They are produced > minutes or hours before the code that they describe is > written. Indeed, they are sometimes produced > concurrently with the code. > > So be careful what you wish for. A switch to agile is not > a switch away from formal documents. You may, in fact, > find that the formality and discipline increases > with such a move. However, you will also find that you > are writing code a lot sooner. ;-)
I don't mind writing documents. I just don't like doing pointless paperwork that no one will read after it's final draft.
> I often hold one-day Agile Overview seminars for company > executives explaining how this works. The remarkable > thing is that they immediately recognize it as they way in > which they run virtually every other part of their > business. Short cycles, lots of feedback, and measurable > results are not foreign to business people. They've just > been told over the years that you can't run software that > way. When they hear that software can be run like > every other part of the business, they get interested.
It sounds pretty good to me. Can you recommend any books?
I find this fascinating. You're a smart guy who knows a lot about agile and (guess it goes without saying) a lot about people. Have you read any of Lakoff's work?
> Are agile > methods leftist because they seem to be egalitarian? Or > are they rightist because they are so full of discipline. > Am I being leftist when I wear my green "Test-First" > " wrist band as a committment that I WILL TEST MY CODE? > Or is that more conservative? If you pair program are > e you also in favor of Gay Marriage? If you practice > continuous integration does it mean you are pro life?
You are misconstruing my intent. I did not mean that Agilistas are politically this way or that, but that the parenting frame strikes me as a good way of distinguishing Agile from, well, let's just call it anti-Agile for the moment. And I think if you will back off a bit and consider it in that light, you will know which is which.
In the Strict Father family, the source of knowledge and power is vested in the Father. The nature of people is to be lazy and evil, so for their own good it is his duty to punish them and keep them on the straight and narrow, so that they will come, in time, to overcome their own laziness that they may one day break away and (in the case of sons) be Strict Fathers of their own family. Society is held together by such disciplined people.
In the Nurturant Parent family, knowledge is distributed. The Parent's role is to protect the children, share his/her knowledge, give the children the opportunity to explore the world in safety and with encouragement. It is understood that children who grow up in safety and (within limits) freedom will themselves come to be nurturant, to care for everyone around them.
Now, then - which of these best maps to Agile, and which to waterfall?
I seem to have pushed one of your buttons, Uncle Bob, and I'm not sorry - I think it's good to explore the issues that most provoke us. But I am fascinated. Thanks for the reply.
I don't think that TDD is necessarily insecure, but it's up to the developer to make sure that security is tested for or it doesn't exist. This includes writing XSS attacks into your tests for websites.
> I need to sit down and write this up at some point, but > the idea has been rattling around my head for a while. > George Lakoff explains the divide between liberals and > d conservatives through the family model frame: liberals > correspond to a nurturant parent model and conservatives > to a strict father model. I believe the same metaphor can > do a lot to explain the split between agilistas and > CMM'ers. And I think the fact that I haven't said who is > which, but you know it anyway, suggest that it's a very > effective metaphor.
I've read a lot of Lakoff, including his non-political stuff. His metaphor has its uses, but like all generalizations, it's both true and not true depending on the particular case. That's what makes it so hard to talk about any of these things. The people who agree see more cases that match and the people who disagree see more cases that don't.
I have another theory, though. I think that the distinction you're drawing is really orthogonal to politics. It's related to the degree to which people place their faith in logic. Hilbert's program collapsed nearly a century ago, but there are still people who feel that you can prove everything.. that the world should be/is an orderly place where all the rules apply.. and damn, it looked so good in the plan.. the plan was so good that it had to work.
I think that as people with engineering backgrounds, we're biased toward this point of view. We're rewarded very early by our computers when we're right and punished severely when we're wrong.
An example: I was at a conference a few weeks ago and a friend of mine was presenting. A member of the audience called her to task for an inconsistency in a notation that she was using. He was very earnest about it and he felt it was serious, but the fact was, it wasn't. It was just an informal notation, and the inconsistency doesn't cause confusion. The notation was a tool and you take it for what it is good for.
I actually, think it is good in this field to overdo it, early on, with formalism and critical thinking. I think that maturity comes when you start to see these things as tools rather than salvation, and you figure out when they apply and when they don't.
I also think it's important to stop believing in the perfectly secure system, and the provably correct system, etc. Reality is thorny. It leaps out of every box you attempt to put it in.
> I also think it's important to stop believing in the > perfectly secure system, and the provably correct system, > etc. Reality is thorny. It leaps out of every box you > attempt to put it in.
That, to me, hit's the nail on the head. This is the main problem with the waterfall model. It assumes everything goes as planned. There is no way (in the process) to address the issues that invariably pop-up at the end of the project.
> This is the main > problem with the waterfall model. It assumes everything > goes as planned.
That view of the waterfall model is the one held only by people wishing to criticize it. In practice, the waterfall model is a series of 'waterfalls' where you don't pass from one stage to the next until you pass the exit tests for that stage, otherwise you loop back and repeat the process. Now there are very many problems with the waterfall model but nevertheless it's wrong t portray it as a one-way ticket.
Flat View: This topic has 35 replies
on 3 pages
[
123
|
»
]
Previous Topic
