On prototyping: software development and the creative process

Last month I wrote about how programming and painting are more alike than people normally suspect. I concluded that there was much we could learn about software engineering by studying how painters work. After all, both disciplines are about the art of making, and both painters and programmers work with a media that is infinitely malleable.

For a long time I have felt that prototyping should be the centerpiece of developing usable software. Why? Because it is only through the process of making that we fully understand the strengths and weaknesses of a proposed design. What looks good on paper often does not withstand the scrutiny of live interaction testing. The primary argument managers of software projects make against prototyping is that it creates throw-away code. But isn't the effort required to create reams of requirements documents and specs also throw-away work (i.e. a source of project overhead)? While it is true that some prototyping work may result is throw-away artifacts, it does not need to be the case.

Returning to my analogy between painting and programming, one might consider a prototype to be similar to an artist's preparatory sketches. Yes, these preparatory sketches are often of little interest after the finished work has been completed (except perhaps to a small cadre of art historians), but they form an important part of the process of making. Why do artists make these sketches? Making preparatory sketches is often part of a research and learning exercise. They enable the painter to explore compositional ideas or to allow him to better understand a subject. While these sketches may be discarded, they are of direct value to creating the final deliverable.

The argument that prototyping is overhead also neglects the possibility for the finished product itself to actually evolve from the prototype. This approach has been legitimized under the heading of evolutionary development, agile programming, and other labels. The core principle of this type of development is that applications should be developed incrementally, with an eye towards delivering software with evolving functionality that is focused on implementing sets of end-to-end user cases. Internally, software should be engineered to be easy to change, and teams should be organized to culturally accept change as a way of doing business.

Again, this feels a lot like the process I follow in making a painting. I've found that each painting develops through a series of incremental evolutionary steps. You make something, you step back and look, you make changes, and you look again. Some painters go as far as to argue that a painting is never "finished." There are only stages of completion along the way. If we take an overall approach to working the canvas, there should be a consistency of resolution throughout. This is the only way one can understand how all the parts relate. Anyone who is familiar with the philosophy of Agile development should notice some parallels here.

If we can recognize some of these common aspects of the creative process, I'm hoping that we can see software development in a different light. I think that the questions we should ask are not how quickly will can complete software design and definition in the interest of getting on with the building, but rather how we can make building and making integral to the definition process. And to see the software life-cycle as a continuum where we are constantly building, looking, and finding ways to futher refine our result.

Painting and programming - strange bedfellows?

I've recently spent a lot of time thinking about the relationship between art and technology since dividing my time as a painter and software designer/developer for the better part of the past 15 years. I used to think of these two activities as separate and unrelated, but now I'm not so sure.

I've discovered that programming and painting have more in common than I could have previously thought. What I've come to understand is that both of these endeavors are about "making." It's about the inherent desire to create artifacts that can solve practical problems, can communicate ideas, or that can inspire. I recently read an interesting article by Paul Graham, entitled Hackers and Painters. Graham holds a PhD in Computer Science from Harvard and also studied painting at the Rhode Island School of Design. He likens the process of programming computer applications, an activity he calls "hacking," to the process of painting. Both he argues are about the desire to create beautiful things. I think he is on to something.

While we might be accustomed to the notion of beauty as it applies to the fine arts, this might be a foreign notion to those who develop technology. But in programming, Graham argues that beautiful software is that which is well designed, and well written. I believe that this idea idea of creating well designed, well made artifacts has inspired craftsman of all types throughout history.

So if programming is like painting then, what does that imply about the way software should be written? I think it suggests that -

• Software, like paintings, are developed through a process of iteration and evolution - that it is impossible to get the design right the first time and that we learn through making.
• Where painters explore ideas through sketching, software developers should explore their ideas through some process of prototyping.
• Great software, like great painting, requires a fanatical devotion to beauty and perfection of the craft.
• Creativity and inspiration are fleeting. It is also important for individuals to find ways to remain productive and work through slumps in creativity that are inevitable to any maker.
• It is important to have empathy with our viewers, patrons, and users. Are we making things that people need? That inspire? That communicate important ideas? We can only know this by engaging with our audience and learning about their activities and desires.
• Finally, what motivates all makers, be they painters, programmers, or other craftsman is the making. We are driven to not just work with ideas but to create tangible things - things that bring pleasure, things that improve lives, things that challenge us to solve problems or to see the world in a new way.

I'd be interested to hear how these ideas relate to the experience of other software developers and artists.