spencegreen.com |
Stuff |
spencegreen.com |
Stuff |
In his eclectic book Let My People Go Surfing: The Education of a Reluctant Businessman, Yvon Chouinard traces the unusual development of Patagonia from a one-man smithing operation in California to the world’s leading producer of outdoor clothing. Chouinard’s self-deprecating style belies his preternatural understanding of the universal human craving for individual freedom. The same impulse that drove him to scale peaks using homemade tools manifests itself in the desire to skip work on Wednesdays or wear unusual clothing. People oppose systems that treat them as cogs. This is one reason for communism’s failure, and it also explains why the assembly line is at once man’s most efficient and least inspiring contrivances:
It is the closest thing to a perpetual motion machine, for its inertia alone seems sufficient to sustain it. In many ways, the modern engineering organization is no different than this assembly line. Whereas Ford has its conveyors and pneumatic arms, the large engineering company has its “Big M” methodologies. Use Python for a business system? Too risky. Compress the management hierarchy? Too controversial. Go on the offensive during requirements development? Too costly. Breaking free from this order takes a refractory personality. This is precisely Chouinard’s conclusion:
One of my favorite sayings about entrepreneurship is: If you want to understand the entrepreneur, study the juvenile delinquent. The delinquent is saying with his actions, “This sucks. I’m going to do my own thing.” Since I had never wanted to be a businessman, I needed a few good reasons to be one. One thing I did not want to change, even if we got serious: Work had to be enjoyable on a daily basis. We all had to come to work on the balls of our feet and go up the stairs two steps at a time. We needed to be surrounded by friends who could dress whatever way they wanted, even be barefoot. We all needed to have flextime to surf the waves when they were good, or ski the powder after a big snowstorm, or stay home and take care of a sick child. Breaking the rules and making my own system work are the creative part of management that is particularly satisfying to me.
Chouinard now has the luxury of reflecting on his ascent, which was fraught with challenges. At one point, he resorted to eating dog food when his money ran out. Such is the life of one who challenges convention, which by definition is a position arrived at by force. A terminal moraine, a huge stone pushed by a glacier, is a natural corollary:
Business, climbing, and even car detailing each have their customs that were developed through natural selection over extended periods. Objecting to an established position is no more palatable than exterminating a particular species of animal. This is why Machiavelli wrote in “The Prince”:
And it should be considered that nothing is more difficult to handle, more doubtful of success, nor more dangerous to manage, than to put oneself at the head of introducing new orders. For the introducer has all those who benefit from the old orders as enemies, and he has lukewarm defenders in all those who might benefit from the new orders.
We often forget that some of our greatest luminaries were not overnight successes, but lately recognized geniuses.
Software engineers do not often have the luxury of designing new systems from first principles. It is frequently the case that they must labor through some dreary chore, such as implementing version 49 of the SuperWhamo! application, or adhering to design constraints imposed not by reason, but by suits. When that rare opportunity to write new code does present itself, two paths are possible. The coder leaps into development, but the architect tries first to solve the problem. This essay contains my observations on the latter approach.
Solve the Problem
Every good system solves a problem, which is often elusive. Think about any famous product and try to describe it with a single sentence or a single image. What does the iPod do? It allows you carry digital audio with you. Google? It lets you find relevant stuff on the Internet. Linux? The world needs a good, free operating system. This is not a pedantic exercise. It took me three months of reading and observation to discover the purpose of a recent project. Not a single person in our organization could articulate the system’s raison d’etre clearly, and I found that when I could, my design work took on a new level of coherence. What does the system do and why does it do it? Why is it useful? Answering these questions can go a long way toward unifying the design process. If the questions can’t be answered, then it might be prudent to get real and kill the project.
Write It Down
Once the system concept becomes clear, write a detailed spec. Specs serve several purposes:
Software specs exist in myriad degrees of formality, breadth, and depth. But the most important thing is that they exist at the beginning:
Writing a spec is a great way to nail down all those irritating design decisions, large and small, that get covered up if you don’t have a spec. Even small decisions can get nailed down with a spec. For example, if you’re building a web site with membership, you might all agree that if the user forgets their password, you’ll mail it to them. Great. But that’s not enough to write the code. To write the code, you need to know the actual words in that email.
I don’t advocate Joel’s informal approach to specs. His method may be sufficient for designing web sites and business systems, but it cannot be used for Space Shuttle avionics or air defense systems. You wouldn’t use the instructions that came with your Coleman tent to build the Empire State building. The IEEE830-1998 standard is a better reference.
Draw It
Any software architect will quickly learn that it is difficult to model a system in its entirety. The software blueprint will probably never exist, a conclusion reached by the Agile crowd a decade ago. Metaformats suffer from a variety of issues, including:
A multi-faceted approach to design seems more prudent. I prefer a combination of tables (for data definitions), sequence diagrams (for modeling interactions between systems), flow charts (for designing processes), schemas (for databases and XML formats) and natural language requirements. If done properly, the latter can be remarkably effective. Like a good specification, an effective natural language requirement should be:
Correct;
Unambiguous;
Complete;
Consistent;
Verifable;
Modifable;
Traceable
Good specs and designs do not guarantee success. As in the entrepreneurial world, good plans do not make rich men. Execution matters.
Build the Organization
This is the most misunderstood diagram in software development:
Systems engineers design the systems that developers implement. Developers should not make judgments about how the user should behave. Likewise, systems engineers should not decide how to implement code. These competing interests need an arbitrator. In this diagram, I have called him an Integrated Product Team (IPT) lead after the Chrysler convention. Microsoft calls him a PM. Whatever his label, he knows enough about systems and software to mediate between the engineering factions. He also understands the business objectives, and can make the difficult distinction between too much schedule pressure, which harms analysis, and too much analysis, which leads to paralysis. He becomes the technical authority, the “System Solon”. Most importantly, he is at the top of the triangle. If software engineering dominates, then cross-cutting attributes such as performance may not be properly evaluated. If systems engineering sets the project tone, then code-level technical insights–”bottom-up” analysis–may be ignored. The Solon is the bulwark against both outcomes.
After reading Steve McConnell’s Software Estimation: Demystifying the Black Art, I called a friend to discuss my newfound insight. Like a child who first learns to write his name, I circled around the central object for no less than 15 minutes. Software is hard in a “different” way! We need statistical methods and mountains of historical data to estimate it properly! Heed these commands or perish! Now my friend is an architect, and she was not moved by this euphoria.
“Are you telling me that your software is more complicated than the Burj Dubai? No one has ever built a building that tall. Moreover, the water table is one meter below the sand, so the whole structure is founded upon a massive concrete pad. Your software has greater complexity?”
Her response stymied me. Some pieces of software exceed that building in complexity by orders of magnitude–the Space Shuttle software, avionics controllers on the Airbus A380, Windows Vista–but how many of us work on those systems? Most engineers slave away on J2EE business platforms, or better yet, “In house” software [link] that solve mundane problems as inelegantly as possible. Not to be stopped, I posited a second argument: software is free from physical constraints, thereby enlarging the solution space. A bridge’s incline, for instance, is limited by the coefficient of friction between the road surface and a car tire: vertical bridges may be possible, but they are not useful.
“Innovation in building has never been more rapid or unbounded,” she countered, “Computer modeling makes unprecedented structures possible today. Although inconveniences like gravity do limit design to an extent, they are probably no more limiting than the constraints imposed upon you by APIs and frameworks. Just look at the Walt Disney Opera House in LA.”
The abstract nature of software is therefore not a reasonable excuse for 99.9% of late software projects. Most of us aren’t “in technology”: we use tools and methodologies that academics developed years ago. You’re not on the bleeding edge. Get over yourself.
I finally advanced an enervated argument based on estimation: it’s hard to finish software on time because software design is difficult to estimate. How long will it take you to finish your math homework? How long does it take to solve a Sudoku puzzle? How long does it take to catch three fish? We’re trying to predict an unpredictable task riddled with risk.
“How long does it take to design anything new? Architects deal with unreasonable requirements, unreasonable customers, and unreasonable deadlines. How long will it take me to design a building down to its moldings and handrails? I’ll tell you how long: a lot of sleepless nights. You can only estimate something if you’ve done it before.”
And this is precisely McConnell’s thesis. Unfortunately, expert opinion is not a sufficient resource. Historical data, on the other hand, has been used in study after study to achieve reliably accurate software estimates. Productivity is an organizational thing, so one organization’s data may not apply to another organization’s projects. Do you think that expert who has worked at 10 different companies can give you a useful estimate based on judgment?
So why are software projects late? I see three problems:
Bad requirements–Ask yourself these questions: does your organization employ a professionally-trained requirements engineer? Hold requirements inspections? Version control requirements at the line-item level? Link those line-items to code? Maintain requirements throughout the entire system lifecycle, including maintenance?
Brooks’s Law–Adding people to a late project makes it later. Graph theory holds the proof to this axiomatic observation. Adding more nodes to a connected graph makes the edge count increase exponentially, not linearly. This is why the scheduling equation to convert staff months to schedule months–the most “agreed-upon” equation in software–has a coefficient and an exponent.
The Iowa Theory–Software engineers are unwilling to do the book-keeping work necessary to make large projects succeed because their brains are trained to look for optimal solutions. Associating requirements with code is tedious, but necessary. Drawing algorithm diagrams is tedious, but necessary. Managing software change is tedious, but necessary. But as software engineers, we think these tasks should be easier. Tools exist to make them effortless–the Telelogic Lifecycle suite, for example–but most organizations don’t invest money in these solutions. So software engineers gripe about tools, and go back to hacking. When we will admit that we’re part of the problem?
Consider the USS Iowa. Her keel was laid down in June 1940 and completed in August 1942. She was 890 feet long, could shoot 1225kg shells over 40km, and could cruise at 30 knots. She was built when the country’s survival was at stake. Does your project have that kind of schedule pressure? She was designed by hand, using pencils, paper, and slide rules. Think about that. All that complexity was managed with filing cabinets, folders, and blueprints. Are software engineers willing to exert that kind of effort?
Powered by WordPress
Entries and comments feeds.
Valid XHTML and CSS.