Combs Spouts Off

Now that the fix is in beta, Microsoft has admitted that Windows has been fundamentally broken for a long time. A long, fascinating article in Australia’s SmartOffice News (based on a Wall Street Journal story) described the revolutionary changes that Microsoft had to make in order to create the next version of Windows:

Allchin is co-head of the Platform Products and Services Division. "It’s not going to work," he told Gates in the chairman’s office mid-2004, the paper reports. "[Longhorn] is so complex its writers will never be able to make it run properly." The reason: Microsoft engineers were building it just as they had always built software. Thousands of programmers each produced their own piece of computer code, to be stitched together into one sprawling program. But Longhorn/Vista was too complex: Microsoft needed to begin again, Allchin told Gates.

It wasn’t a message Gates wanted to hear. The article describes how Jim Allchin and his associates, Brian Valentine and Amitabh Srivastava, became convinced that Microsoft had to scrap the half-finished Longhorn project and abandon the existing Windows code base, how they then convinced Bill Gates and Steve Ballmer, and how that decision has changed the architecture of Windows, the way code is written at Microsoft, and even the way the company is organized.

Allchin’s group realized that the Windows code had become increasingly difficult to patch, update, build, and test due to its complexity and interdependencies:

Ideally, engineers make a fresh build every night, fix any bugs and go back to refining their features the next day. But with 4,000 engineers writing code each day, testing the build became a Sisyphean task. When a bug popped up, trouble-shooters would often have to manually search through thousands of lines of code to find the problem.
…
… A newcomer to the Windows group, Mr. Srivastava had his team draw up a map of how Windows’ pieces fit together. It was 8 feet tall and 11 feet wide and looked like a haphazard train map with hundreds of tracks crisscrossing each other.

The group decided that Windows should be built more like UNIX/Linux OSes, with a small, robust "kernel" providing core functions and everything else a Lego-like plugin module. They also insisted that Microsoft’s traditional coding practices had to give way to a more disciplined approach, with more testing, fewer bugs, and less last-minute heroics:

While Windows itself couldn’t be a single module — it had too many functions for that — it could be designed so that Microsoft could easily plug in or pull out new features without disrupting the whole system. That was a cornerstone of a plan Messrs. Srivastava and Valentine proposed to their boss, Mr. Allchin. Microsoft would have to throw out years of computer code in Longhorn and start out with a fresh base. It would set up computers to automatically reject bug-laden code. The new Longhorn would have to be simple. It would leave bells and whistles for later — including Mr. Gates’s WinFS, Messrs. Srivastava and Allchin say.

The new plan was adopted in August 2004, although Gates apparently remained reluctant and skeptical, and dramatic changes were made rapidly:

By late October, Mr. Srivastava’s team was beginning to automate the testing that had historically been done by hand. If a feature had too many bugs, software "gates" rejected it from being used in Longhorn. If engineers had too many outstanding bugs they were tossed in "bug jail" and banned from writing new code. The goal, he says, was to get engineers to "do it right the first time."

To address Gates’ concern that the team was too focused on process and not paying attention to what the engineers thought, Srivastava brought on board the "revered elder statesman" of Windows engineers, Dave Cutler:

On Dec. 1, Mr. Srivastava escorted Mr. Cutler to Microsoft’s auditorium where the software guru told 1,000 engineers that he had used the tools to build Windows code that was nearly bug-free. That Mr. Cutler — famous for never attending meetings — would emerge to back Mr. Allchin’s revolution helped persuade some engineers to drop their objections.
…
As engineers began cooperating and Mr. Srivastava’s team worked overtime to refine the tools, the quality of the code flowing into Longhorn began to improve. The time to create a new "build" fell to just a few days, allowing a faster cycle of writing and testing new code. After the Windows group was able to install a workable version of the system on their PCs four days before Christmas, Mr. Srivastava says the group celebrated by not working over the holidays.

It hasn’t all been rosy since. The beta of Longhorn, now named Windows Vista, wasn’t shipped to 500,000 customers for testing until July 27, over a month late. But the number of reported problems was an order of magnitude smaller than what past experience had predicted. And apparently everyone, including Gates, is now a believer:

It could take years before Windows can be as flexible as Microsoft needs it to be to pump out new features quickly. But the cultural shift is in swing. Hours after showing off Windows Vista to software makers this month, Mr. Gates in an interview noted how Microsoft’s Office group is now using some of Mr. Srivastava’s tools to improve its code. "It’s amazing the invention those guys have brought forward," he said. "I wish we’d done it earlier."

It’s remarkable that Microsoft, given its size, profitability, and market dominance, was able to overcome institutional inertia and undertake such fundamental, revolutionary changes. It speaks well of Gates, the top management, and the corporate culture. 

In January 1998, Virginia Postrel had an article in Reason magazine called "Creative Insecurity" that serves as an excellent bookend to the above story. Postrel, a user and fan of Apple computers, explained why Microsoft became the dominant, standard-setting software company even though many of its products have been less than stellar:

… Great products did not make Microsoft number one. Good-enough products did.

That uncomfortable truth offends moralists on both sides of the Microsoft debate. The company’s fans (and its spin doctors) want to tell a simple tale about virtue triumphant–with virtue defined, Atlas Shrugged-style, not only as astute business decision making and fierce competition but also as engineering excellence. Its critics use the same definition. If the products are less than great, they suggest, the only way to explain the company’s success is through some sort of sleaze. Or, alternatively, through the innate flaws of the market.

So what really happened? How did Microsoft end up ruling PC operating systems and, through them, software in general?

At the risk of simplifying a complex story (if only by reducing it to two players), the bottom line is this: Apple acted–and continues to act–like a smug, self-righteous monopolist. Microsoft acted–and continues to act–like a scrambling, sometimes vicious competitor.
…
… This is not a company that thinks like a monopoly. It is always running scared. There’s always the possibility that something new could come along and destroy its franchise.

Microsoft is still running scared in 2005. In the 1990’s, it was scared of Apple and Netscape. Today, it’s the Linux/open-source community and Google. Worthy competitors with fine products in each case. But as long as Microsoft continues to run scared and not act like a monopolist, I like its chances. And I suspect I’ll like Windows Vista.

UPDATE: For a different perspective, check out Linda Seebach’s column in the Rocky Mountain News. She’s been brainwashed by her UNIX-loving son, Peter, who she quoted as saying Microsoft has "genuinely evil intent" (I can’t tell if that’s tongue in cheek or serious). So she’s not exactly persuaded by my conclusion. 🙂

Subscribe To Site:

Full Post Feed | Summary Feed | Comments Feed