Today I Wrote a Functional Specification
But not for the normal reasons...
The Manifesto for Agile Software Development has a value:
Working software over comprehensive documentation
It’s a way of expressing a priority: our job is primarily to deliver value to the customer, and 99% of the time the expression of that value is the customer using the system you deliver.
As with all the values in the manifesto, it’s expressed as a preference, and not as an absolute. If I prefer tea over coffee, then I’ll likely choose tea all other things being equal. But sometimes I have to have coffee because I’ve run out of tea; sometimes I just feel like a change.
Unfortunately, some folks choose to interpret the four manifesto values as rules. “It says ‘Working software over comprehensive documentation’,” they say, “So we don’t have to write any specs.”
When I talk to folks about this value (which, to be honest, isn’t that often), I don’t say “write this document” or “write this code.” Instead, I ask if they are ways of writing the code that preclude the need for the document. And I give them an example from my own dim and distant…
The Telex Switch
Back in the 1980s I had a software business in the UK. We landed the contract to implement the system to route incoming international telex traffic to its final destination.
A brief aside for any readers not yet collecting a pension: Telex was a system where text messages were typed out, normally onto paper tape. They were then placed into a reader connected to a phone line by a modem. The receiver of the message was dialed on a telephone, and, when the connection was made, the contents of the paper tape were encoded in various whistles and beeps1 and sent down the line, where they were converted back into text and printed. Telex messaging worked great when you needed a physical record, or when do couldn’t rely on the other person being present.
Although initially point-to-point, telex traffic eventually became automated. It was routed depending on the destination, and often aggregated down higher bandwidth trunks before getting split out onto slower-speed local lines.
Now back to the tale…
Our software was to perform that switching for telexes arriving on trunk lines from abroad.
Naturally the client (a national telephone company that shall not be named) were very concerned that the software did what it was supposed to, handling errors while delivering messages to the correct destination without loss.
Our contract called for us to deliver a test specification before any code was written. The customer provided dozens of examples of possible messages, along with a paragraph describing what our code should do.
Test-Driven Documentation
Our first task was to take their English language explanations of how messages should be handled and convert them into something more formal: divert message to error bin 27; forward message over local POTS connection, and so on. These actions were the outputs our software should generate when fed the message.
We put this into a document that was hundreds of pages long. The client read it, signed off on it, and we started coding.
Eventually, the time came for them to run acceptance testing. They turned up at our office and said they’d budgeted three days to run these hundreds of tests. They figured it would take that long to type in each message and then manually check the resulting output.
We pulled our rabbit out of the hat. The original document we gave them containing the test descriptions was actually generated directly from the test code. We showed them how this worked, and demonstrated that the translations were accurate. We then issued the command that ran the tests, and the printer started documenting the results, in the same format used in our original document.
I have no idea if they actually went back to their office that afternoon, or whether they took the rest of the three days as personal time…
And the Point Is…
For me, this story encapsulates the manifesto value. The most important thing was the passing tests. But the initial, human readable document, was essential, too. We honored the manifesto value (that was still 20 years in the future) by making the code the primary expression of the tests, and using exactly that code to generate the documentation needed by the client.
For what it’s worth, this is also an illustration of the DRY principle (also almost 20 years away). DRY stands for Don’t Repeat Yourself, and the principle is:
Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.
We expressed the knowledge of the tests in code, and then ran that code with one of two output processors, one producing troff for the documentation, and the other producing C for the tests themselves.
I would never claim documentation is evil. But I do believe that producing most documentation is a waste of developer time, particularly if that documentation is only needed to fulfill some step in a project management doctrine.
Turn the Page
Documentation is a tool. Like all tools, it has to earn its place in your toolbox. I know, many of you will have mandated documentation deliverables. The question you might want to ask your cost-conscious manager is “How do we measure the value we get out of this report?” They may well talk about intangible long-term benefits. Cool. So what has been measured for all the older projects? Can they still even find that documentation?
Having said that, I’m actually a fan of good documentation. Pictures that show me the lay of the land before I start diving in to code are fantastic (as long as they’re up-to-date). Short lists of nonfunctional requirements can help explain the decisions that were made. And, sometimes, writing a spec is a great way of clarifying your thinking.
So Why Did I Write A Functional Spec?
I’m in the middle of rewriting the Pragmatic Bookshelf website and the corresponding back end systems. For the front end we’re using as much off-the-shelf software as we can, which means that when it comes time to analyze sales data to calculate royalties, we have to take a feed in their format and convert it into something our existing royalty software can use.
On the surface, royalty calculations are easy. You sell a book for $30, the author gets a 50% cut, so we add $15 to their royalty account.
In practice, they can be tricky. For example, a customer accidentally buys a book they already own; we give them a refund. If that happens before the monthly royalties are run, we can just correct the amount in the list of pending transactions. But if we don’t issue the refund until after we’ve paid royalties, we have to generate a new royalty transaction to correct the first.
It gets even more complicated when an order has multiple items, and where coupons are involved.
Two days ago, I sketched a few pictures, outlined a handful of database tables, and started coding; I thought I could see my way through. Today, I finally decided that that sick feeling in my stomach was my subconscious telling me I was messing things up. I’d already made one false start, and it was becoming obvious that I was on my way to my second.
So the dog took me for a walk, and I tried to work out how to get my head around what I was trying to do.
When I got back, I fired up Google Docs and started writing what was effectively a functional specification. I tried to write it for some random developer who’d end up implementing it. And, as the words and diagrams flowed, everything became a lot clearer. When I found myself struggling to write about something, I’d stop and ask myself why. Often it was because the thing I was describing was either totally unnecessary or ridiculously over engineered. When I tried to make the diagrams look nice, I found new relationships between things that were previously disjoint.
It took perhaps 90 minutes, And now I feel I know how to write this code.
Thinking about it, the exercise was a a lot like rubber-ducking.2 As I was writing, I was focussing on explaining my ideas to some future third party. When I found myself thinking this has to be done this way, I imagined my reader asking why? Everything was now open for exploration; implicit assumptions were made explicit.
I’m stopping for the day. Tomorrow, I’ll scan through the document I wrote, delete the code from my latest attempt, and start again. The likelihood is that I’ll never look at the document again.
It’s just a tool; it served its purpose, solved a problem, and will now become obsolete.
The start of a dial-up session (although this is more modern than the modems used by Telex machines…
When you get stuck, you explain your problems to the rubber duck sitting on top of your screen. Somehow, the act of externalizing the issue makes things clearer.
Thanks for sharing that, Dave. I guess it's the mechanization of the process that makes things bad. Not questioning "why" we have to write a document and just doing it instead of understanding the purpose of that document to find strategies so we can create whatever artifact we need without wasting efforts.