A challenge that comes with writing down experiences occurs when writing about events that readers lived through, have strong opinions about, and feel they know the full story. My purpose here is to share what we were thinking and doing at the time, how a broad set of people reacted, and then to offer my views of the reasons leading to the results. That is a way of saying this section is going to start with what we set out to do, not where we ended.

Back to 101. Reimagining Windows from the Chipset to the Experience: The Chipset [Ch. XV]

We set out to reimagine Windows, but it is interesting to ask what exactly the product is to most people. This helps us to understand the thoughts behind making a major change. We saw this in the changes to Office—we recognized the value of Office was not as many might have thought, the file formats or the old File|Edit|View|Tools|Window|Help menu structure, but rather the inherent capabilities and implementation of those capabilities. In that spirit, we looked at Windows and saw much more than the specifics of the Start menu or any particular expression of a user interface. Windows at its core proved to be a remarkably resilient operating system and our goal was to tap that resiliency to bring it new capabilities for a new world of customers.

Fundamentally an operating system can be thought of as the software that allocates hardware resources, manages connectivity and devices, and defines the human to device interaction model. Beyond these technology distinctions Windows is also a culture of openness to developers and an ecosystem of partners that itself has proven resilient over time. Windows is no more one of those than a single feature or attribute above others.

As both a participant in and later a contributor to the evolution of Windows, I find the transitions that the Windows product has gone through to be a case study in the “soft” part of software and in the flexibility of the product team to engineer transitions from one evolutionary stage to another. Consider just some of the transitions the Windows OS kernel or core operating system have undergone:

* GUI transition. Windows itself was the product of a transition that many doubted could be made.  Could an entire OS be built upon the "shaky" underpinnings of MS-DOS in the face of Macintosh?  A remarkable amount of work went into the technology across the ecosystem, such support for 32-bit microprocessors, that made Windows 3.0 such a watershed product.  Yet underlying that, customers could still bring forward investments in all those applications and peripherals from MS-DOS.

* 32-bit transition. The transition to 32-bits was one that required a vast amount of change and brought with it the introduction of "plug and play" and the ability to run more sophisticated graphical applications and games of unrivaled qualities. The introduction of Windows 95 was a watershed moment for the whole industry. While in hindsight it looks as though it was a sure thing, many at the time proclaimed it would be technically impossible.

* Internet transition. Immediately after the release of Windows 95, the conventional wisdom quickly became that Windows would be replaced by a browser.  Yet few would have argued with the fact that the presence of Windows—the support for networking, the introduction of graphical web browsers on Windows, as well as the openness of the platform all contributed to the transformation of the world of information technology.  In 2010 we were just starting to see how the powerful graphics of Windows could bring standards based HTML5 to life in unprecedented ways. As we will see in the next section, the programming model and API of Windows was indeed still struggling.

* Server Scale. As we continued to evolve the client (workstation) OS we were using this same OS foundation to power the datacenter.  With Windows Server we scaled the OS to support hundreds of computing cores and terabytes of storage.  And along the way we created this OS for multiple CPU architectures driven by the demands of server computing (Alpha, MIPS, Itanium, and 64-bit).  At each step most people believed that such flexibility was neither prudent nor possible.

* Security and Reliability. Through the above transitions, there was an undercurrent that Windows was "aging" and that it could not transition to modern computing needs of much larger memory architectures, multi-core OS support, improved reliability, and much better security.  The introduction of Windows XP was a milestone in bringing our enterprise server and workstation OS to mainstream consumers.  Ironically, at the introduction of Windows XP many thought we had reached too far, and that the OS was more than people would need or could even afford to power. Throughout this transition the introduction of Windows Update enabled the OS to stay connected to customers, provide code updates, and distribute software on behalf of the ecosystem.

These transitions were supported by consistently strong engineering efforts to refactor, rearchitect, and re-tune the Windows operating system. Windows also showed a broad set of efforts at the experience or user interface layer in the system as well. The various editions of including Windows Home, Windows Professional, and Windows Enterprise while often viewed as pricing and licensing efforts did in practice introduce a wide range of functionality tuned to different customer segments. Windows was able to reach up and down in complexity using the resiliency and flexibility inherent in the architecture.

Yet, at the experience level Windows also struggled to achieve critical mass in several important areas even with the capabilities of the team and assets in the code. In Hardcore Software, we’ve seen the difficulties of creating tablet computers (Tablet PC), handheld computers (Windows CE), home entertainment (Windows Media Center), industrial devices (Windows Embedded), and most recently the ongoing development of smartphones (Windows Phone 7.)

What is it about the experience layer that has proved so difficult for Microsoft? There could be many possible explanations. Was Microsoft too early when it should have been patient? Was Windows code simply the wrong starting point? Did we bring out good products but had inferior marketing, sales, and distribution? Some would point to one of more of these.

I had my own theory and plan for trying something different.

My view was that Microsoft relied too heavily on the notion of architectural layering and believed that experience could always be layered “on top” of the operating system. This computer science view drove nearly every discussion on how things should move forward. It was viewed as they key to architecting Windows for these different experiences while maintaining the shared code of the actual operating system.

Microsoft systems culture (aka BillG) loved to believe that with the right architecture things in one layer could advance independent of another. I don’t think I could begin to enumerate all the times we debated issues that boiled down to me suggesting the abstractions are in the way while Bill insisted that great architecture would solve the problem. The counter was that I was failing to consider building a great architecture before addressing the problem. I was merely suggesting that we might end up spending 90% of the time on 10% of the problem.

In my heart I am an “apps person” as I’ve noted many times. One thing that is conceptually different about apps from operating systems is that apps tend to be much less hardcore (or religious as some might say) about architectural layers while much more zealous about solving the customer problem even if that means breaking through well-considered abstraction layers. In an operating system there’s a general tendency to view the layers and architecture as goals the system must achieve and moving forward the layers enable advances and are deeply respected. Whereas in apps the layers and architecture tend to be the starting point while innovation and moving forward usually involve breaking those very abstractions.

Each of the above attempts at expanding the experience of Windows were implemented with the explicit goal and architectural starting point of not interfering with the core of Windows. Importantly, the teams executed without organizational alignment across the Windows team or schedule. The essence of the former COSD division was to serve the main missions of Windows for desktops and for servers but generally favored the server mission for cultural and historic reasons. Whereas the Client division served the multiple experiences of Windows but did not generally prioritize experiences beyond the main client. To be clear I am not saying this was wrong at the time, rather it introduced tradeoffs that had side effects relative to other missions.

This to me was the kind of decision made early in a project that is difficult to work around. The inability of the Tablet PC to fully embrace native Windows implementations or for Windows Phone to tap into the available power of the Windows operating system support for devices and connectivity were examples I’d personally experienced. As an apps person in the “two gardens” sense, the same way we needed Word and Excel to share code and align around everything from the use of the Windows registry to HTML to drawing code to user interface, we needed Windows to align around these expansions and changes to the experience.

Windows was making a tradeoff at eac