Turning average programmers into good programmers

It is difficult to hire the most productive programmers in the world. Oftentimes these people are capable, by themselves, of turning out entire products, and thus they start their own companies. If a really productive programmer works for an established organization, that organization will usually take extreme steps to keep him or her. Thus beyond a certain point it is most effective for an organization to develop a strategy for creating good programmers internally.

How does one create a good programmer? Raw materials are important. You want someone with a strong computer science education, a high IQ, and an ability to communicate effectively in oral presentations and in writing. But without the right experience, such a person will never be more than an average quality programmer.

These principles are important in building up someone's programming skills:

1. A person won't become proficient at something until he or she has done it many times. In other words., if you want someone to be really good at building a software system, he or she will have to have built 10 or more systems of that type.
2. A person won't retain proficiency at a task unless he or she has at one time learned to perform that task very rapidly. Learning research demonstrates that the skills of people who become accurate but not fast deteriorate much sooner than the skills of people who become both accurate and fast.
3. Technology shifts force a programmer to go through bursts of learning every year or two.

Look around your organization. You can make a list of the people qualified to design and build a new system by counting up those who've built 10 or more similar systems in the past, at least two in the last year, and that could do the entire job in a month or two if they really had to. These are your "good programmers". Everyone else is a candidate to be turned into a good programmer as quickly as possible.

Learning to design and build software systems requires that the programmer design and build software systems. These can be smaller subprojects for internal or external customers, standalone software system for non-profit organizations, or demonstration systems to be written up and distributed to other programmers. A particularly effective option that is only available in the Web world is to build and launch a free public service. See http://remindme.arsdigita.com and http://towzone.arsdigita.com for examples of one-evening training projects.

Whatever the training task, the pace must be ruthlessly brisk. The learner should be expected to build at the same pace as an experienced developer. The difference between the learner and the wizard is that you expect the learner to make a lot of mistakes. The system as built may be awkward or not handle error cases properly. That's okay. Training research shows that if you get speed now you can get quality later. But if you don't get speed you will never get quality in the long run. We practice this technique in 6.916, Software Engineering for Web Applications, our course at MIT. Each student builds five database-backed Web applications during the 13-week semester. The first few that they build, during the course of the problem sets, are not necessarily elegant or optimal, but by the end of the semester they've become remarkably proficient, especially when you consider that each student is taking three or four other classes.

If you see one of your best people walking out the door at 6:00 pm, try to think why you haven't challenged that person with an interesting project. If you see one of your average programmers walking out the door at 6:00 pm, recognize that this person is not developing into a good programmer. An average programmer's productivity will never be significant in a group of good programmers. If you care about profits, you must either come up with a new training program for the person or figure out the best way to terminate his or her employment with your organization.

Still not convinced? Take a look at the Japanese "code factory" circa 1990. These precisely organized large organizations where each person had his role, however small, were supposed to overtake the American approach where small teams of craftsmen worked in a comparatively disorganized manner. The factory approach sometimes produces acceptable corporate IT solutions but for innovation and successful product development, the craft approach has been overwhelmingly vindicated.

Turning good programmers into good managers

As noted in the introduction, software engineering is different because the organization can't afford to lose the individual productivity of the best people as they are pushed into management. At ArsDigita, for example, a manager who is one or two levels up from the leaf nodes is still expected to write code, develop SQL data models, write system design documents, and write journal articles. Yet managers who are spending a portion of their time designing software or writing documentation are at risk of neglecting their duties to review subordinates' work.

The classic problem situation at ArsDigita is a manager getting lost in his or her own work and failing to review a subordinate's efforts for two or three months. When the review occurs, inevitably the subordinate has either been working on the wrong thing in the wrong way or hasn't been sufficiently productive. At this point the manager is really angry. Three months of calendar time and money have been wasted. But should the manager be angry with the employee? If the manager had reviewed the subordinate every week, the company would never have been at risk of losing more than one week of time and money.

Our solution is to decouple responsibility for review from responsibility for scheduling review. We use administrative assistants to ensure that each manager is scheduled to look at every subordinate's work at least once per week, more frequently in the case of junior employees. It has proven remarkably more effective when a neutral third-party is responsible for scheduling than when people with incentives to shirk are responsible for scheduling.

Management by Consensus Considered Harmful

Leaf-node engineers at every company on this planet think that they have better business ideas than the senior managers. Why not simply turn the company over to the engineers to run? Each engineer has a different set of better business ideas.

Software engineering companies will tend to have a fairly flat distribution of intelligence. The 22-year-old Stanford CS punk that was just hired will be just as smart as the 30-year-old lead engineer who will be just as smart as the 40-year-old CEO. Within a company's technical team, the raw IQ differences are even smaller. If each member of the team were playing the Bach Partitas and Sonatas for Solo Violin, the wrong notes, shaky intonation, and bad phrasing would make it pretty obvious to the novices that they needed to take advice from the experts. But because software quality is tough to measure and software quantity is seldom measured, the novices in a software engineering group are able to think of themselves as experts.

What would be wrong with a completely egalitarian software engineering group? Maybe the entire team really is at the same level of ability. And suppose that somehow the challenge of getting everyone to attack the same problem had been surmounted. Remember what Fred Brooks said in The Mythical Man-Month: high quality systems must be architected by no more than two people.

Getting design input from leaf-node engineers is important for having a good product design. But at the end of the day nobody should be confused as to who is providing leadership. There is an irreducible amount of Engineer A imposing his or her design on Engineer B. This can lead to some harsh-sounding words and bruised feelings. Microsoft is not the self-esteem company, at least if you believe Playboy magazine's interview with Bill Gates: "We hear you're brusque at times, that you won't hesitate to tell someone their idea is the stupidest thing you've ever heard. It's been called management by embarrassment challenging employees and even leaving some in tears." Truly elite organizations can be far worse than Microsoft. Ask a group of surgical interns and residents how much respect they get from the surgeons. Go into a world-class biology department and ask the grad students and post-docs about their treatment at the hands of the professors.