Java Community News - Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Articles |
News |
Weblogs |
Books |
Forums

Artima Forums | Articles | Weblogs | Java Answers | News

Sponsored Link •

Java Community News
Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

38 replies on 3 pages. Most recent reply: Jun 14, 2008 10:09 AM by John Zabroski

Welcome Guest
Sign In

Back to Topic List

Reply to this Topic

Search Forum

Threaded View


Previous Topic		Next Topic

Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 ]

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 9:07 AM

> Given this disparity, it seems
> crazy that, at least in my experience, the amount of
> effort applied to documenting the plan is so much more
> than the effort put into documenting the final system.

Spot on. Savvy project managers will plan time for documentation after implementation. However, much of the time re-assignment to other projects occurs and that planned time is considered a luxury instead of a necessity. I'd like one solid week after implementation to produce design, run, etc. documents.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 9:53 AM

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 2, 2008 10:13 AM

> In my world, project managers couldn't care less about
> what happens after implementation.

Unfortunately, even the ones who do care and "get it" are pushed by those above them to get to work on the latest and greatest project, meanwhile, the just implemented project is old news. I'd be willing to bet that they often have incentives to get X, Y, and Z projects implemented so they don't have the incentive to let each one be completed properly.

Mark Thornton

Posts: 275
Nickname: mthornton
Registered: Oct, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 3, 2008 1:52 AM

"a small percentage of the world’s population is good at programming, and a small percentage is good at writing; apparently I am asking everybody to be in both subsets"

It is worse than that as you also need to be comfortable with the markup scheme. I rather like TeX as it is compact and quite comprehensive, but most of my colleagues do not. The 'modern' approach might be to use XHTML+MathML+SVG, but this markup is unsuited to writing by hand in a regular editor. So perhaps we need IDE support as well, but I haven't seen any ...

Eric Gillespie

Posts: 13
Nickname: viking
Registered: Jun, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 3, 2008 3:42 PM

> > Given this disparity, it seems
> > crazy that, at least in my experience, the amount of
> > effort applied to documenting the plan is so much more
> > than the effort put into documenting the final system.
>
> Spot on. Savvy project managers will plan time for
> documentation after implementation. However, much of the
> time re-assignment to other projects occurs and that
> planned time is considered a luxury instead of a
> necessity. I'd like one solid week after implementation to
> produce design, run, etc. documents.

To turn things back around somewhat, what happened to the write documentation first, then write the source from the documentation? If something needs to be added (and who knows of a program where it doesn't?) then document it first?

As I understand it, this was one of the prime tenets behind literate programs like weave/tangle.

Also, how does this contrast with the "only document what's needed" mindset that most programmers end up in, because there's literally no time to do anything else? At what point does practicality kick in?

Bill Pyne

Posts: 165
Nickname: billpyne
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 4, 2008 7:06 PM

> To turn things back around somewhat, what happened to the
> write documentation first, then write the source from the
> documentation? If something needs to be added (and who
> knows of a program where it doesn't?) then document it
> first?

As with most things in life, it depends on the situation. Try doing that in a Boston area startup company and the company will be out of business in a heartbeat. I worked in a company along the 128 belt sometime around 1995. They had precisely 2 weeks worth of reserves to pay employees in case a sale didn't come through. When you're running on that kind of margin you need to have turnaround within days of a sale. When I worked for a finance company, of middling size, they bought and sold smaller companies on a monthly basis. It was all finance, but the models were all different. A company bought one month might be sold by the next quarter. Documentation was a waste in that environment. I can concede that a company with an already solid business and a relatively static business model would benefit from the document first approach. Repeating what I said at the top, it depends on the situation though.

> As I understand it, this was one of the prime tenets
> behind literate programs like weave/tangle.

As I mentioned in another response within this thread I have no knowledge of Literate Programming but feel it would be worth my time to read up about it. I'll take from it what I feel is worthwhile and leave the rest behind.

> Also, how does this contrast with the "only document
> what's needed" mindset that most programmers end up in,
> because there's literally no time to do anything else? At
> what point does practicality kick in?

I hate arguing against a fictitious group. I'm not willing to concede that most programmers end up in that mindset.

My experience is that documentation is devalued by the organization. Managers are willing to talk at great lengths about the importance of it but I haven't experienced many who were willing to postpone implementation dates so that solid documents could be written.

From my experience alone, I think that "living" documents are the only worthwhile ones. By this I mean documentation in the form of wiki's. Documentation should be a simple process devoid of any back-and-forth in committee approvals. I'm in the middle of that now. Sometimes it's a week before I hear back from a single committee member. By that time, I've lost interest in what I'm doing.

Practicality is very relative depending on your mindset. You'll have to be more specific about what you think is practical in this instance.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 4, 2008 7:42 PM

> To turn things back around somewhat, what happened to the
> write documentation first, then write the source from the
> documentation? If something needs to be added (and who
> knows of a program where it doesn't?) then document it
> first?

You haven't specified the detail of the documentation so any response is going to be based on the reader's assumptions of what you mean.

But to give an example where I think the above doesn't make any sense, I can take an example from my current work. One group I work with requires that all designs be documented to a level of detail such that 'any developer can code from them'. In other words, these documents basically contain the same level of detail as the code itself. These documents also take roughly the same amount of time to write as the code itself. Effectively, this means the code is written at least twice: once in Word and once in the target language. Any changes compound this duplication of effort.

While there is value in the document in that it's easier to read than the code, especially if you aren't familiar with the language, most of the useful detail could be generated from the code automatically.

Basically what you are suggesting is exactly what a lot of shops force on their developers. And yes, you can do it that way but you are going extremely inefficient (i.e. slow). Where I work, this problem manifests itself as unhappy customers. Their expectations and needs are not met by this approach. Often the final product arrives long after it is no longer needed.

> Also, how does this contrast with the "only document
> what's needed" mindset that most programmers end up in,
> because there's literally no time to do anything else? At
> what point does practicality kick in?

What's the argument against "only document what is needed?" What's the value in writing documents that are not needed?

Javier Diaz Soto

Posts: 29
Nickname: javierbds
Registered: Sep, 2005

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: May 9, 2008 7:38 AM

I think the commercial world of sw has changed a lot in the last 20 years. To clients, sw is no longer some strange incantations to make machines behave. As a result, clients expect sw to be at the same time: flexible, optimizable, evolvable, ...
Clients expect to see some results sooner and, more often than not, are satisfied with prototype quality sw. Clients do not know o do not have the time to know: what when or how they want. They also will not take what you give them and ask for what they see as "little" changes. So if you make a lot of work up front you WILL throw it away. Clients will not let you nail them down on a solution. And clients don´t care how it looks inside as long as it does what they want. As most development is externalized, they just see sw as its interface.
Those who pay the price are developers that have to deal with all the crap that this type of process generates: bad, out-of-synch docs, architecture that has rot, ad-hoc design, code replication, functional replication, high churn of devs ... (and I am leaving away political-organizational issues). If sw project could, in general, be taken as seriously as other engineering efforts we would not have reached this state of things.

I see XP as an adaptation on the part of the people that make the products to the Real Practices of the world. Some see it as a cause of the above problems, but I see it as a reaction. Most of what goes into XP is not "experimental", it is adaptation and evolution as targets move.

John Zabroski

Posts: 272
Nickname: zbo
Registered: Jan, 2007

Re: Donald Knuth on Multi-Core, Unit Testing, Literate Programming, and XP

Posted: Jun 14, 2008 10:09 AM

> In my view, literately programmed code
> should be visible for every programmer
> through its evolution, so literate
> programming doesn't cope with OCP well.
> I guess it is why he prefer "re-editable"
> code to reusable black box code.
> But I think benefits of information
> hiding should not be sacrificed.

I prefer to take a more formal view. There are four levels of specification: defined, implementation defined, unspecified and undefined.

You should be able to re-edit implementation defined facts. The problem is the model you choose to write your programs in gets translated into an application binary interface that necessitates encapsulation. However, as Robert Young could probably point out, encapsulation is not nearly an important principle as data independence. In fact, the ability to break encapsulation while mathematically preserving functionality leads to optimizations! Sometimes the abstractions preserved by encapsulation are fundamentally too coarse-grained to be re-usable and shareable in complex systems. Moreover, in many systems today, encapsulation is ill-defined and results in ambiguous copy and transfer semantics: In C++, the compiler needs to know if an object embedded inside another object is owned by the object instance conceptually or merely related to the object instance. However, the proposed constraint solver for C++0x will fix this somewhat. Furthermore, many systems do not define guarantees between the caller and the callee, such as exception safety, so valuing their encapsulation is dubious at best if you don't know what the input and output wires map to.

Finally, anything unspecified complicates understanding of a system. Some systems have formal semantics that are complete, and so it is impossible for the system's configuration space to contain an unknown configuration.

Flat View: This topic has 38 replies on 3 pages [ « | 1 2 3 ]

Previous Topic

Next Topic