On prototyping: software development and the creative process

Last month I wrote about how programming and painting are more alike than people normally suspect. I concluded that there was much we could learn about software engineering by studying how painters work. After all, both disciplines are about the art of making, and both painters and programmers work with a media that is infinitely malleable.

For a long time I have felt that prototyping should be the centerpiece of developing usable software. Why? Because it is only through the process of making that we fully understand the strengths and weaknesses of a proposed design. What looks good on paper often does not withstand the scrutiny of live interaction testing. The primary argument managers of software projects make against prototyping is that it creates throw-away code. But isn't the effort required to create reams of requirements documents and specs also throw-away work (i.e. a source of project overhead)? While it is true that some prototyping work may result is throw-away artifacts, it does not need to be the case.

Returning to my analogy between painting and programming, one might consider a prototype to be similar to an artist's preparatory sketches. Yes, these preparatory sketches are often of little interest after the finished work has been completed (except perhaps to a small cadre of art historians), but they form an important part of the process of making. Why do artists make these sketches? Making preparatory sketches is often part of a research and learning exercise. They enable the painter to explore compositional ideas or to allow him to better understand a subject. While these sketches may be discarded, they are of direct value to creating the final deliverable.

The argument that prototyping is overhead also neglects the possibility for the finished product itself to actually evolve from the prototype. This approach has been legitimized under the heading of evolutionary development, agile programming, and other labels. The core principle of this type of development is that applications should be developed incrementally, with an eye towards delivering software with evolving functionality that is focused on implementing sets of end-to-end user cases. Internally, software should be engineered to be easy to change, and teams should be organized to culturally accept change as a way of doing business.

Again, this feels a lot like the process I follow in making a painting. I've found that each painting develops through a series of incremental evolutionary steps. You make something, you step back and look, you make changes, and you look again. Some painters go as far as to argue that a painting is never "finished." There are only stages of completion along the way. If we take an overall approach to working the canvas, there should be a consistency of resolution throughout. This is the only way one can understand how all the parts relate. Anyone who is familiar with the philosophy of Agile development should notice some parallels here.

If we can recognize some of these common aspects of the creative process, I'm hoping that we can see software development in a different light. I think that the questions we should ask are not how quickly will can complete software design and definition in the interest of getting on with the building, but rather how we can make building and making integral to the definition process. And to see the software life-cycle as a continuum where we are constantly building, looking, and finding ways to futher refine our result.