The Software Practitioner Triad
by Alan Cooper

Enterprise Application Lifecycle Management 2003

Alan Cooper
Agree? Disagree? Let us know by e-mailing us at .    The opinions expressed in this editorial are those of the author and not necessarily the opinions of VSM.

Today, Web designers are called programmers, programmers are called engineers, engineers are called architects, and architects never get called. Not only are our titles mixed up, but our community of software practitioners is also deeply confused about the roles we play. The confusion is even worse in the minds of the businesspeople who hire us and set our budgets and schedules.

In my last column, I showed that programming—constructing release code—isn't the same as engineering (see Additional Resources). Production programming is as demanding, difficult, and creative as engineering, but its primary goal is producing a shippable product, not solving its technical problems. Shippable programs are mostly protective code that rarely—if ever—executes. Hundreds of thousands of lines of
code are dedicated to supporting obscure edge cases, platform idiosyncrasies, and error conditions. The code must then be tested vigorously and thoroughly. These demands multiply many times the cost in both time and money of any backtracking or last-minute changes to release code. As Frank Lloyd Wright said, "You can use an eraser on the drafting table or a sledgehammer on the construction site."

On the other hand, technical problem solving demands experimentation, which is naturally repetitive and empirical. This means that the tasks repeat, and failure is as educational as success. But failure and repetition are antithetical to production's nature. Conversely, for the engineer's iteration and experimentation to be fast, accurate, and cost-effective, it can't be fettered with the demands of producing release code.

Clearly—for the sake of the schedule, the budget, and the customer—programmers should never be tasked with engineering duties, and engineers should never be directly responsible for programming release code. But even if a shop establishes these two roles correctly, a key piece of the complete solution is still lacking.

The panoply of software construction includes three vital roles: the programmer, the engineer, and the architect. The architect is responsible for determining who the user is, what he or she is trying to accomplish, and what behavior the software must exhibit to satisfy these human goals. The engineer's responsibilities are comparable but focused on technology. A good engineer can and should ignore human issues, confident that the architect will cover the human side.

The bulk of the role confusion arises from the simple fact that few software projects are architected. Instead, the programmers and the engineers try—with limited success—to fill in the gap. What's more, the title itself is often confused with the obsolete and ill-named "software architecture," which is advanced software engineering done close to the metal.

Iteration is Endemic
Anybody who's worked around software knows that no one gets it right the first time. The creation of anything computerized inevitably involves lots of trial and error, backtracking, tearing out, and rebuilding. This rebuilding is fractal in nature: Individual lines of code are rebuilt, separate procedures are rebuilt, whole objects are rebuilt, entire programs are rethought, platforms are discarded and reselected, human interfaces are recoded completely, and even the program's concept and market get morphed into unrelated areas. It doesn't take a rocket scientist to see that such cyclic repetition is crushingly expensive, and simple mistakes can cascade into huge, costly, career-limiting blunders. Everyone works to reduce or mitigate this costly and risky iteration.

Engineers tend toward making the iteration go away by detailed advance planning. Programmers prefer to reduce the time allotted to each iteration and perform the iteration at the last possible moment. An unfortunate fact is that these approaches are mutually exclusive: One can't do detailed advance planning and then do equally detailed daily or weekly modifications, without driving up cost and blood pressure. This is why engineering methods such as Rational Unified Process don't coexist well with Extreme Programming techniques.

Many executives deny the need for iteration altogether, sometimes demanding to ship prototype code to avoid the necessary rewrite. Other executives have institutionalized the idea of shipping a slapped-together, first approximation of a product, anticipating that their customers will tell them which changes are needed to make the product a success. What's more, these same executives have come to expect that their customers will pay for each new iteration, driving businesspeople to demand frequent, big, remunerative iterations—once a year, for example—but to eschew the myriad little iterations that merely enhance quality. Either of these methods can be effective in uncompetitive markets, but they're deadly over the long term, primarily because they trample on customer loyalty. Clearly, the most expensive possible moment to iterate is upon completion and delivery.

The key to driving down the cost of iteration isn't to eliminate or delay it, but to isolate and concentrate it where it helps and to make it unnecessary where it's expensive and ineffective. In practice, this means that iteration should start at the earliest possible moment and finish soon thereafter, well before any release code is even begun. Iterating before coding means each cycle is fast, simple, and inexpensive, and doesn't cause domino-falling changes in other existing code.

Engineering and production programming in most software shops are undifferentiated, and architecture isn't done at all. This means that all the necessary technical and human iteration are mixed into the daily round of production coding, where iteration is dauntingly expensive and can be eliminated.

Two changes must occur: Engineering must be differentiated from programming, and architecture must be added. The proper sequence of events begins when the architect identifies who the users are (or will be) and what their goals are. The architect can then rough out the program's form and behavior (more on this later).

User Needs Trump Technical Issues
Concurrently with the architecture, the engineer researches the technical landscape to determine how the product should best be constructed to run most efficiently in the target environment. The engineer designs a technical specification that satisfies the project's technical goals. However, these technical questions can't be answered in a vacuum. User issues always trump technical ones. Users must get their problems solved, and failing to solve them with more-advanced technology isn't better than solving them adequately with older mechanisms.

Of course, the architect and engineer must work together closely, because the software's behavior will depend to some extent on the technology's limitations, and the technical specifications will be influenced strongly by the human goals it addresses. (One frequent, fatal mistake is to assume that technical issues are separate from "user interface" issues, and that the "back end" can be created independently from, and in advance of, the "front end.")

The good news is that the architect works on a whiteboard or on paper, and the engineer works with tiny snippets of test code. Neither of these makes the same arduous demands that production programming does, so the iterations are relatively fast and inexpensive, and will have no domino effect on production schedules or budgets.

By far, the questions that demand the most iteration in software are the human ones. Every programmer knows the misery of delivering a finished product to end users, only to have them whine and complain about how "it isn't what I wanted," even though it is exactly what they asked for. This is the problem that architects solve. Their research and design methods allow them to see beyond what the users say and determine what they really mean, even though the users themselves are unaware of the truth.

The architect researches the user community to determine who will use the product and what they must accomplish with it. The architect then designs a form and a set of behaviors that satisfy the users' goals. He accomplishes this by making a first approximation of an interaction design solution, and then improving it iteratively through a user-modeling technique involving the use of hypothetical but representative user archetypes, called personas. This iteration is accomplished without code.

Only after the architects and the engineers have determined in precise detail what will be built, and how it will be built, can the production programmers begin to build it. The programmers can focus exclusively on the special and difficult demands of creating fast, bug-free, maintainable, shippable code, because all the gross experimentation has been completed.

The biggest obstacle to building software properly is fear. Programmers are afraid of losing challenge and creativity, engineers are afraid of losing control, and executives are afraid of failing. Many programmers worry that architects and engineers will usurp the creative, challenging part of their job, leaving them merely to code what they're told. They worry that their jobs will degenerate into mindless drudgery, devoid of intellectual stimulation. Nothing could be further from the truth.

Engineers and Programmers Share Skills
The challenge of creating production software is enormous, and the creative skills it requires are vast. Engineers take none of this away from programmers; they merely focus on more-generic, performance-related problems. Engineers' and programmers' skill sets are often identical. Their temperaments are where they differ. Both like to solve puzzles and to build things, but programmers are most satisfied by seeing their work completed and shipped. Engineers are most satisfied by simply solving problems, whether or not they are part of a useful object. It's an obsolete holdover from the industrial age to think that engineers are somehow better or smarter than programmers.

The perpetual battle between engineers and programmers over waterfall methods vs. rapid iteration are purely artificial, caused largely by the lack of form and behavior design. The architect, of course, provides clear and exhaustively detailed descriptions of the program's form and behavior that she can prove will satisfy the users' goals. The programmer-engineer battles cease, because the need to grope for interaction solutions disappears.

Programmers who believe an architect will suck all the stimulating, creative parts out of their work and leave them merely to code according to the architect's plans have never worked with a real architect. My architecture-consulting company encounters this fear frequently, but upon completion of our work, the client programmers are unanimous in declaring that our efforts helped make their work more challenging and creative. Programmers, by nature, love solving problems creatively in directing the computer to perform as desired. Also by nature, they don't like solving problems in human desires, but this is precisely what architects enjoy and are good at. Our clients are all relieved and astonished to find that the problems we address are typically ones they weren't even aware existed. What's more, once the architect has identified the users' goals and designed behavior to support them, the programmer's tasks become even more challenging and creative than ever before.

Most of us geeks are motivated by technical excellence more than by business success. Executives sense this, so they distrust us and keep us on far too short a leash. The only way to get the time and freedom we need is to earn their trust, and we can do that only by bringing in people with the proper skill sets and interests and having them work together to create successful products. If we are ever to be treated with the respect we deserve, we must get our job roles straightened out. Conflating programming and engineering, and asking either role to do user-focused architecture, sets our cause back. The artificial title inflation that dogs our industry isn't helpful, and the confusion it generates in the business community that hires and finances us is poisonous. Let's put an end to the internecine fighting between programmers and engineers by bringing in architects and letting them define software's form and behavior before any code is written.

About the Author
Alan Cooper heads the software-consulting company Cooper, where he has spent a decade making high-tech products easier to use and less expensive to build, a practice known as interaction design. Get the skinny on Alan at or send him your polite, intelligent, copyedited e-mails at .