Here is a very interesting talk from Douglas Crockford on software quality, and in particular, why all software seems to be buggy, over-budget and late. He suggests reasons why this appears to be an endemic property of all software.
In addition to what Crockford says, I’ll add that software programming is fundamentally unpredictable. Either you are doing something you’ve never done before, or you are trying to figure out why something won’t work. There is very little about programming that is routine and predictable.
Note: Discussion continues below.
Literate Programming
Crockford talks about Literate Programming which is a very interesting idea which just hasn’t received any traction. The basic idea is:
- The documentation and the specification are embedded in the code.
- The program is designed to be read by a human.
And I’m not talking about code comments and JavaDoc, which merely documents the functions and classes in the program. I mean literal prose like “The purpose of this program is to provide a service for teachers to keep track of student marks (grades) as they try to complete their project”.
Literate programming is difficult, especially on a larger team, and there really aren’t many tools available to do it. But since I am a sole programmer I can make the rules, and I decided to try it out as an experiment. My most recent project is a markbook for teachers to keep track of marks (grades) associated with Ingot Certificates.
The experiment
The first step was to choose a documentation tool since that’s the cornerstone of Literate Programming. Long story short, I chose Natural Docs. I wanted something more “standard” like oXygen or JavaDoc, but no other tool has nearly enough flexibility for what Literate Programming requires. With reason, these tools are designed to write APIs, not software specifications.
Natural Docs is a surprisingly flexible system, almost wiki-like, where you can write a lot of interesting high-level documentation, with headings, sub-headings, lists, character formatting, embedded source code, etc. I do wish it had tables, embed HTML and other features. So it is not at the level I’d like for true Literate Programming, but it’s pretty good. In addition, the ND syntax is very natural looking, so the program source code itself is still easy to read.
How did the experiment turn out? Quite well actually. Approaching the project in this way really helped me think about the problem in ways that I would not have otherwise. Often I found myself struggling to explain why something was designed in a certain way, and I realized that this meant that the design was not as logical as it seemed at first. Then I would come up with a more logical design and end up with a much better product as a result. The experience was interesting, often frustrating, but also rewarding. Would I do it again in my next large project? Definitely.
