Open Sources 2.0/Beyond Open Source: Collaboration and Community/Making a New World

From WikiContent

< Open Sources 2.0 | Beyond Open Source: Collaboration and Community
Revision as of 19:06, 5 May 2008 by Docbook2Wiki (Talk)
(diff) ←Older revision | Current revision (diff) | Newer revision→ (diff)
Jump to: navigation, search
Open Sources 2.0

Doc Searls

Open Sources was published in January 1999. That same month I became a full-time editor for Linux Journal, assigned to cover Linux in business. Over the next seven months, I also co-wrote (with Chris Locke, David Weinberger, and Rick Levine) The Cluetrain Manifesto, a rant that first took the form of a web site and later took the form of a book. Its subtitle was "The End of Business as Usual."

Cluetrain, like much of what I wrote for Linux Journal at the time, argued against bubble-headed marketing at a time when bubble-headed investing in "Linux companies" was growing to galactic dimensions. In August 1999, Red Hat had the largest IPO run-up in stock market history. In December 1999, VA Linux (now VA Software) set a new record, which remains unbroken. In January 2000, Cluetrain hit the bookstores and became a business bestseller. The bubble began to pop on January 17, and within a few months nearly everybody in a "Linux business" (including Linux Journal) suffered consequences ranging from dire to fatal. All businesses, including Linux ones, still feel the after-effects.

Yet the bubble was a red herring. It was off-topic in the extreme, because Linux was quietly being put to use in businesses everywhere, along with a growing suite of other open source infrastructural building materials. Linux was never about the stock market, or even about business. It was about something else—something that caused usage (especially in business) to grow regardless of whatever happened among commercial suppliers.

Today the spotlight is on a new set of Linux business leaders: IBM, HP, Novell, Oracle, Red Hat, and other large companies. We know they are leaders because they buy big booth space at LinuxWorld, pay employees to work on Linux code, make good products, sell to the growing market demand for Linux and open source goods, do good PR, and get lots of coverage in the press.

No offense, but they're red herrings too. They matter a lot, but they're not what Linux is about. They're not what open source is about, either, because Linux and open source are demand-side developments. They are all what the demand side does to supply itself.

Just like the environment where open source took root and grew: the Net.

If designing and building the Internet had been left up to the usual suspects, it never would have happened. Networking would still be a private affair, a grace of large vendors, each operating their own separate and barely interoperable networks. For an example of what that would be like, consider instant messaging. IM is a network service that never found its way into the Internet suite (unlike, say, hypertext, file transfer, email, and domain names). The situation for IM is not much different today than it was back in the 1980s, when "online services" like Compuserve, AOL, and Prodigy each had their own incompatible email systems. Today Yahoo!, Microsoft, AOL, and Apple all remain committed to closed proprietary IM systems that run on the Net, but are not of the Net, meaning they contribute nothing to the Internet's open, free, shared, and ubiquitous infrastructure. They are "platforms" supporting closed silos that trap and hold dependent inhabitants.

The platform and silo system is as old as computing. It's still with us, and won't go away quickly, if ever. But as a defining model for the software business, it is being replaced by a growing assortment of open standards and open source tools and building materials that together support far more business than they replace. Linux and its familiar LAMP suite (Linux, Apache, MySQL, PHP, Perl, Python, PostgreSQL, etc.) are the most obvious ones. SourceForge lists another 90,000, with dozens more added every day.

Nearly all of these tools and building materials were created by the demand side of the marketplace, to solve practical problems, and to provide useful infrastructural support for similar activities. The free and open way they contribute to the world is good for business. Whole businesses and business categories, old and new, are sited on bedrock composed of open standards and open source components. Growth horizons for these businesses and categories are unobstructed by dependencies on vendors and their platforms. Their environment is the wide world of the Net, not the inside of some vendor's silo. In fact, all platforms and silos in the computer business now find themselves in a subordinate position to the Net. To survive they have to operate in the wide new world of the Net.

That goes even for Microsoft, which built the largest and most widely used platforms and silos in computing history, with a monopoly in the most ubiquitous product category of all: personal computing.

Yet even Microsoft finds its vast monopoly, and all of its platforms and silos, forced to live in a new and larger world that:

  • Nobody owns
  • Everybody can use
  • Anybody can improve

In "World of Ends" (http://www.worldofends.com), David Weinberger (http://www.hyperorg.com/blogger) and I initialized those three principles as NEA.

In the first two respects, this virtual world is like our physical one. Except for the patches of crust we call real estate, nobody owns the Earth or its atmosphere. And everybody can use the Earth's base infrastructural provisions: gravity, air, and filtered sunlight.

Exceptions can be found, of course. For example, MySQL is owned. Yet, as infrastructural building material, it doesn't matter whether MySQL is owned or not, because everybody can use it and anybody can improve it. MySQL's ownership matters only to those that choose to have a commercial relationship with the company. In that respect, MySQL doesn't behave as a traditional "platform" vendor, trying to lock customers and third parties into a dependent role in a private environment. Instead, it behaves more like a provider of building material for a construction project. MySQL, like other open source components, is modular stuff. It's made to work inside a larger context—a job, a design, an architecture, whatever—where each component does its job and not much more than that.

The list of open source building materials grows constantly. When I started writing for Linux Journal, there were a handful of familiar names: Linux, Apache, Sendmail, and Perl. That grew to become the LAMP suite (where the M represents MySQL and the P includes PHP and Python). Now the names dropped in a discussion of open source might also include Tomcat, Squid, Asterisk, JBoss, Eclipse, Jabber, ZeroConf, RSS, iPodder, or any of the 90,000+ projects on SourceForge alone.

That's because NEA's third principle—anybody can improve it—gives humans the power to continue making this new world. Think of this principle as do-it-yourself geology. We don't just play God here. We get to do His job.

The architecture of this world was first described in 1983 by J.H. Saltzer, D.P. Reed, and D.D. Clark in End-to-EndArguments in System Design (http://www.hyperorg.com/blog). Fourteen years later, TheRise of the Stupid Network, by David Isenberg, delivered a death sentence to the conceits of network centralizers. The Stupid Network was an end-to-end argument against AT&T's cherished belief in The Intelligent Network. David wrote, "A powerful leading indicator of the Stupid Network will arrive when entrepreneurs who have no vested interest in maintaining telephone company assumptions begin to offer profitable, affordable, widely available data services." A prophesy now fulfilled.

Craig Burton (http://www.craigburton.com) combines both ideas—end-to-end and stupid—by describing (http://www.linuxjournal.com/article/4158) the Internet as a hollow sphere, composed entirely of ends:

I see the Net as a world we might see as a bubble. A sphere. It's growing larger and larger, and yet inside, every point in that sphere is visible to every other one. That's the architecture of a sphere. Nothing stands between any two points. That's its virtue: it's empty in the middle. The distance between any two points is functionally zero, and not just because they can see each other, but because nothing interferes with operation between any two points. There's a word I like for what's going on here: terraform. It's the verb for creating a world. That's what we're making here: a new world. Now the question is, what are we going to do to cause planetary existence? How can we terraform this new world in a way that works for the world and not just ourselves?

One current example is podcasting. The term first appeared in August 2004. When I wrote about it in IT Garage (an online sister to Linux Journal) in September 2004, a search for podcasts brought up 24 results on Google. Now the same search gets 778,000 results. By the time you read this, the number will probably be in the millions.

Podcasts are audio files distributed to subscribers' audio players (mostly iPods) via Really Simple Syndication (RSS), an XML dialect designed to serve as a syndication format for weblogs. The prime movers behind podcasting are Adam Curry and Dave Winer. Adam is a veteran broadcast personality (best known as an early MTV VJ) and a serial entrepreneur. Dave is a programmer, writer, and businessman whose fingerprints are on XML-RPC, SOAP, OPML, outlining, blogging, and other useful innovations, including RSS.

Podcasting grew rapidly after Adam created iPodder (http://ipodder.sourceforge.net), a script that automatically routes podcast feeds into iPods and other MP3 players, via computers. As a distribution system for audio (or any kind of media file), iPodder and its relatives are forming an infrastructural foundation for a whole new industry—one in which anybody can participate. And, by providing a limitless supply of talent, material, and low-cost any-to-any distribution, podcasting also offers boundless new opportunities for broadcasting, cable, satellite TV and radio, the record business, and lots of other industries—as well as to noncommercial institutions ranging from churches and civic organizations to public broadcasting and government.

Note that podcasting became a hot category without the help of a large company. Instead, it began with the demand side supplying itself.

Now watch for big companies to jump in, and for businesses of all sizes to start making money. And watch for most of that money to be made because of podcasting's open standards and open source components, instead of with them.

It will eventually become clear to everybody that far more money is being made because of open source than with open source. This is what we have to remember every time somebody asks, "How can you make money with (open source product)?" The answer is, "You don't make money with it. You make money because of it."

The because of principle is old hat in mature business categories, but it's new to the software business. Too many of us still want to see "business models" for all kinds of goods that don't belong on the income sides of balance sheets. Would you ask your telephone what its business model is? How about your front porch? Your driveway? Your clothes? Those things may help us make money; but they are not how we make money. Well, the same goes for open source products. They are a means to an end. You make money because of them, not with them.

It's also easy to forget that the most original sources in this new world are not technologies, but talented and productive human beings. We all know reputation is tremendously important in hacker culture, and that open source is required, literally, to substantiate reputations. It is less obvious that the same is true for every other talent that operates on the Net. Reputation grows fastest when the goods and services of creative minds are open to inspection, improvement, adoption, and reuse.

Take the case of English Cut (http://www.englishcut.com), the weblog of Thomas Mahon, bespoke Saville Row tailor, London. As I write this, Mahon's blog is a couple of weeks old, and he already has a Technoraticosmos (collection of current inbound links) north of 200 (up from 100 yesterday), which is remarkable (http://www.technorati.com/cosmos). Thomas Mahon is quickly becoming the most authoritative Saville Row tailor in the world—not because he's a terrific tailor (which he surely is), but because he also operates in the public marketplace we call the World Wide Web. There he converses with customers and fellow mavens about fabrics, drafting, cutting patterns, the trade, and (of course) his advantages over competitors. Thomas Mahon runs an open source business. Literally.

Not surprisingly, he has already attracted all the business he can stand. Again, because of his blog. Not with it.

Thomas was urged into blogging by one of his customers, Hugh MacLeod (http://www.gapingvoid.com), a cartoonist and professional marketing iconoclast whose blog is Gapingvoid.com. Hugh took the free ideas delivered by The Cluetrain Manifesto, and leveraged them into HughTrain (http://www.gapingvoid.com/Moveable_Type/archives/000823.html), a collection of wisdom that adds substantially to the open source-savvy marketing canon.

Thomas and Hugh are both referenced in this piece of wisdom (doc.weblogs.com/discuss/msgReader$5489) from Mike Warot (bitgrid.blogspot.com), which appeared in the comments section of my own blog (doc.weblogs.com) this morning:

I think that division of labor and specialization are the ultimate human skill set. I see the invention of blogs, wikis, and other online communication as the most recent additions to the toolbox. We're going to increasingly bestow rewards based on competence (as the Hughtrain and English Cut weblogs have done), but it won't be a direct reward.It's going to be especially interesting to watch how the online expression and validation of trust and competence evolve. Cory Doctorow hinted strongly at this with the concept of Whuffie. I've seen, secondhand, how a friend of mine gets small but pervasive and persistent rewards, for having been the first to engineer a simple, open source, file-transfer protocol. There's not big money in it, but had he tried to make it proprietary, it won't have flown, and something else would have taken its place. My opinion is he's responsible for enabling a big chunk of the PC revolution through many tangents.He invented something, gave it away (purely for the fun of it), and got some Whuffie in the long run.Scary thought: we're going to see Whuffie become quantified (and even perhaps monetized) at some point. We have to plan for that eventuality.This model will take the place of marketing for companies that are able to articulate and share their knowledge with others. I particularly find the English Cut weblog to be fascinating. I've learned quite a bit about the culture and technology of well-made suits. Should I happen to work my way into wealth, I might even become a customer. It certainly has increased my appreciation of the value of a Bespoke. He's adding a lot of value from my perspective.

Reputation matters. Authority matters. Google (perhaps the world's biggest example of how to make money because of open source) sorts search results by the PageRank (http://www.google.com/technology) system, which the company explains in this way:

PageRank© relies on the uniquely democratic nature of the Web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important."

That importance is what Cory Doctorow called Whuffie (en.wikipedia.org/wiki/Whuffie), in his book Downand out in The Magic Kingdom. By whatever name—reputation, authority, brand value, Whuffie—we don't acquire it alone. Its value is bestowed by others. In fact, the same might be said for its substance.

Several years ago I was talking with Tim O'Reilly about the discomfort we both felt about treating information as a commodity. It seemed to us that information was something more than, and quite different from, the communicable form of knowledge. It was not a commodity, exactly, and was insulted by the generality we call "content."[1]

Information, we observed, is derived from the verb inform, which is related to the verb form. To inform is not to "deliver information," but rather, to form the other party. If you tell me something I didn't know before, I am changed by that. If I believe you and value what you say, I have granted you authority, meaning I have given you the right to author what I know. Therefore, we are all authors of each other. This is a profoundly human condition in any case, but it is an especially important aspect of the open source value system. By forming each other, as we also form useful software, we are making the world, not merely changing it.

Stewart Brand provides a helpful framework for understanding that world, with this "Layers of Time" model of civilization, from TheLong Now (http://www.longnow.org), as shown in Figure 15-1.

Figure 15-1. Brand's Layers of Time

Brand's Layers of Time

Look at this as a layered section of surface on Craig Burton's hollow sphere—this "World of Ends" we call the Net.

At the bottom we find the end-to-end nature of the Net. It's also where we find Richard M. Stallman, the GNU project, the Free Software Foundation (FSF), and hackers whose interests are anchored in the nature of software, which they understand fundamentally to be free.

When Richard M. Stallman writes "everyone will be able to obtain good system software free, just like air," he's operating at the Nature level. He doesn't just believe software ought to be free; he believes its nature is to be free. The unbending constancy of his beliefs has anchored free software, and then open source development, since the 1980s. That's when the GNU tools and components, along with the Internet, began to grow and flourish.

The open source movement, which grew on top of the free software movement, is most at home one layer up, in Culture. Since Culture supports the Governance, the open source community devotes a lot of energy and thought to the subject of licensing. In fact, the Open Source Initiative (OSI) serves a kind of governance function, carefully approving open source licenses that fit its definition of open source. While Richard and the FSF, sitting down there at the Nature level, strongly advocate one license (the GPL or General Public License), the OSI has approved around 50 of them. Many of those licenses are authored by commercial entities with an interest in the governance that supports the infrastructure they put to use.

In fact, it was an interest in supporting business that caused the open source movement to break off of the free software movement. That break took place on February 8, 1998, when Eric Raymond wrote "Goodbye, `free software'; hello, `open source'" (http://www.catb.org/%7eesr/open-source.html). Here is where the Culture layer can clearly be seen moving faster, and breaking from, the Nature layer:

After the Netscape announcement broke in January, I did a lot of thinking about the next phase—the serious push to get "free software" accepted in the mainstream corporate world. And I realized we have a serious problem with "free software" itself.Specifically, we have a problem with the term "free software" itself, not the concept. I've become convinced that the term has to go.The problem with it is twofold. First, it's confusing; the term "free" is very ambiguous (something the Free Software Foundation's propaganda has to wrestle with constantly). Does "free" mean "no money charged" or does it mean "free to be modified by anyone," or something else?Second, the term makes a lot of corporate types nervous. While this does not intrinsically bother me in the least, we now have a pragmatic interest in converting these people rather than thumbing our noses at them. There's now a chance that we can make serious gains in the mainstream business world without compromising our ideals and commitment to technical excellence—so it's time to reposition. We need a new and better label.I brainstormed this with some Silicon Valley fans of Linux (including Larry Augustin of the Linux International board of directors) the day after my meeting with Netscape (Feb. 5th). We kicked around and discarded several alternatives, and we came up with a replacement label we all liked: "open source."We suggest that everywhere we as a culture have previously talked about "free software," the label should be changed to "open source." Open source software. The open source model. The open source culture. The Debian Open Source Guidelines. (In pitching this to the corporate world I'm also going to be invoking the idea of "peer review" a lot.)And, we should explain publicly the reason for the change. Linus Torvalds has been saying in "World Domination 101" that the open source culture needs to make a serious effort to take the desktop and engage the corporate mainstream. Of course he's right—and this re-labeling, as Linus agrees, is part of the process. It says we're willing to work with and co-opt the market for our own purposes, instead of remaining stuck in a marginal, adversarial position.This re-labeling has since attracted a lot of support (and some opposition) in the hacker culture. Supporters include Linus himself, John "Maddog" Hall, Larry Augustin, Bruce Perens of Debian, and Phil Hughes of Linux Journal. Opposers include Richard Stallman, who initially flirted with the idea but now thinks the term "open source" isn't pure enough.Bruce Perens has applied to register "open source" as a trademark and hold it through Software in the Public Interest. The trademark conditions will be known as the "Open Source Definition," essentially the same as the Debian Free Software Guidelines.It's crunch time, people. The Netscape announcement changes everything. We've broken out of the little corner we've been in for twenty years. We're in a whole new game now, a bigger and more exciting one—and one I think we can win.

Seven years later, victory is all but complete. And not just for open source. While the free software movement has been, relatively speaking, a drag on the open source movement, the deep and abiding beliefs and commitments of free software advocates have anchored both movements and have helped software identified by both labels succeed.

Not coincidentally, the Culture on which this new world depends is hacker culture, about which Eric S. Raymond—a founder of the OSI—has written extensively (he edited both editions of The Hacker's Dictionary). Both he and Bruce Perens, another leading open source figure, have purposefully advocated open source to business for many years.

And although open source hackers tend to be more interested in business than free software hackers, both want Governance and Infrastructure that support business but are not determined by business—except when business works with the hacker community. Hence OSI's license-approval process. While the number of open source licenses has been a source of some debate (almost everybody would rather see fewer licenses), it is important to note that the relationship between these layers is not the issue. The last thing anybody in the free software or open source movements wants is for anybody at the Commerce level to reach down into Governance to control or restrict Infrastructure that everybody relies upon. Even though that's exactly why large companies, and whole industries, hire lobbyists. More about that issue shortly.

Changing corporate culture to adapt to open source development methods is not easy. Dan Frye, who runs IBM's Linux development program, recently told me that IBM has worked hard to make its internal development efforts coordinate smoothly with Linux's. That way, when IBM "scratches its itches," the kernel patches that result have a high likelihood of acceptance. IBM has faith that its accepted patches are ones that are most likely to work for everybody and not just for IBM. This is a natural and positive way for infrastructure to grow.

And grow it has. The selection of commodity open source building materials is now so complete that most businesses have no choice but to use those components—or, in many cases, to recognize that IT personnel in their enterprises have been building their own open source "solutions" for some time.

That realization can come as a shock. Open source infrastructure inside companies often (perhaps usually—it's hard to tell) gets built without IT brass knowing about it. In many cases, internal open source development and use has had conditional approval by CIOs and CTOs. Whatever the course of open source growth, at a certain point a threshold is crossed, and companies suddenly know that open source is no longer the exception, but the rule. The result for IT is often something like "Oh God! Our sex has changed! What do we wear?"

Several years ago, when I showed the diagram in Figure 15-1 (along with others we'll visit later) to Rob Glaser, founder and CEO of RealNetworks, he made a remarkable observation: that the Internet revolution rocked the business world because, for the first time in history, Infrastructure changed faster than Commerce. "It was like the rug got pulled out from under everybody," he said.

On the one hand, this caused a great deal of excitement, some of which gassed up the dot-com investment bubble. On the other hand, it caused a great deal of fear, some of which got lobbied into the Digital Millenium Copyright Act (DMCA). In the first case, commercial interests were enthused by the new Internet infrastructure but failed to understand its deeper causes and principles. But at least they did no harm to the Nature, Culture, or Governance that produce and support Infrastructure. Not so with the entertainment industry and the lawmakers it successfully lobbied for passage of the DMCA. The DMCA is a prime example of how Commerce can lead Governance to screw up Infrastructure.

The DMCA was intended mostly to protect copyright holders from the ravages of Nature in a networked world where, as Richard Stallman put it in the GNU Manifesto (http://www.gnu.org/gnu/manifesto.html), "Copying all or parts of a program is as natural to a programmer as breathing, and as productive. It ought to be as free."

While its authors wanted the DMCA to protect their industries by limiting "piracy" (their word for illegal copying) of copyrighted works, in practice the DMCA has chilled free speech and scientific research, thwarted "fair use" (an established copyright permission), slowed the protected industries' adaptation to life in the networked marketplace, and crippled or prevented new business categories from emerging.

The case of Internet radio is instructive. The DMCA defined broadcasting on the Net as "performance," and digital copies as "perfect," regardless of their fidelity. It required "webcasters" to negotiate royalties with the recording industry, through a Copyright Arbitration Royalty Panel (CARP) administered by the U.S. Copyright Office. Led by the Recording Industry Association of America (RIAA), the CARP instituted royalty requirements so labyrinthine, difficult, and costly to webcasters that it effectively prevented the industry it purported to regulate. Had it been imposed on over-the-air broadcasting at the dawn of that industry, it would have strangled that baby in the cradle too.

One unintended consequence is podcasting. Today podcasters are growing in fertile markets that haven't been poisoned by the DMCA. "Podsafe music" (also known as "non-RIAA" music) is already a staple on podcasts. Once podcasting becomes sufficiently popular (and is perceived by business as something happening below the Fashion layer), look for the RIAA either to take advantage of it or to force leading new podcasting companies (or established broadcasting companies) to strike Faustian bargains with them, just so those companies can play RIAA-licensed music. Whatever happens, the DMCA's market poisons cripple or prevent far more business than it protects.

The Commerce level is inherently proprietary (you can't have business without property); but that doesn't need to be a problem for open source, or for anybody. That's because, in a mature and healthy industry, property claims are limited to each owner's own property. If you want to build platforms and silos, fine. Just don't expect to support whole categories, because there are lots of open and free infrastructural building materials already doing that.

The construction industry provides a good model for where we're going here. There are plenty of platforms and silos among construction materials and methods. For example, windows from Andersen, Eagle, and Marvin require their own branded replacement parts, or parts from "compatible" manufacturers. But since construction is a mature industry, with a countless variety of commodity materials made compatible by widely accepted standards, no one manufacturer can "own" the industry, or even a part of it, by popularizing a "platform" on which everybody else is required to build. It would be absurd to build a house on the "Weyerhaeuser platform" or the "Georgia-Pacific platform." Building materials are essentially modular. All your studs, joists, siding, wallboard, cinder block, roofing, flooring, lighting, and electrical materials are provided by manufacturers whose proprietary concerns generally don't extend beyond their own products and the purposes to which they are put.

Open source is a natural quality of most construction. Materials and methods are open to inspection, and copying. If a builder finds a better way to put up shingles, hang a door, nail a joist, or pour a foundation, they don't keep it a secret. They share it, and the knowledge gets passed along from crew to crew.

It's also true that many construction materials are full of patents and other intellectual property claims. Yet those claims are limited in scope by the modular nature of construction standards and practices. For example, you may choose to use proprietary door latches from one manufacturer throughout your house. Those latches may contain inventions patented or licensed by the manufacturer. But that manufacturer wouldn't think of extending those claims to the doors that contain those latches or to the whole house. To do so would be worse than absurd; it would be self-defeating.

It's not a coincidence that we already talk about software in terms of construction. We have "architectures," "builds," "designs," tools," "frameworks," "levels," "platforms," "components," and "structures." We have "sites" with "addresses and "locations" that are often "under construction."

The similarities between software and construction are so close in some ways that we can't help making sense of the former in terms of the latter. As cognitive science puts it, construction is a conceptual metaphor for software development and use. Lately George Lakoff (http://www.rockridgeinstitute.org/people/lakoff), the father of cognitive linguistics, has also done some borrowing from the same source. Instead of talking about "conceptual metaphors," he now talks about "frames" and "framing" (http://www.rockridgeinstitute.org/projects/strategic/simple_framing). In Patterns of Software (Oxford, Paperbacks, 1996), Richard Gabriel says, "Habitability is the characteristic of source code that enables programmers coming to the code later in its life to understand its construction and intentions and to change it comfortably and confidently." His ideal example is the New England farmhouse:

The result is rambling, but each part is well suited to its needs, each part fits well with the others...The inhabitants are able to modify their environment because each part is built according to the familiar patterns of design, use and construction and because those patterns contain the seeds for piecemeal growth.

Stewart Brand says the same kind of farmhouse is an ideal example of "vernacular" construction:

What gets passed from building to building via builders and users is informal and casual and astute. At least it is when the surrounding culture is coherent enough to embrace generations of experience.

Vernacular is a term borrowed since the 1850s by architectural historians from linguists, who used it to mean the native language of a region. It means "common" in all three senses of the word—widespread, "ordinary," and "beneath notice."

In terms of architecture, vernacular buildings are seen as the opposite of whatever is "academic," "high style," or "polite." Vernacular is everything not designed by professional architects—in other words, most of the world's buildings. Vernacular building traditions have the attention span to incorporate generational knowledge about long-term problems such as maintaining and growing a building over time. High-style architecture likes to solve old problems in new ways, which is a formula for disaster.

Vernacular buildings evolve. As generations of new buildings imitate the best of mature buildings, they increase in sophistication while retaining simplicity.

The opposite of vernacular is what Stewart calls "magazine" architecture: artistic, idealized, expensive, and made to impress rather than to operate. Here's another difference, described by Henry Glassie: "If a pleasure-giving function predominates, it is called art; if a practical function predominates, it is called craft."

It's hard to imagine anything more crafted and practical than a command-line interface. That insight comes to us from Neal Stephenson, the novelist and hacker, in his book Inthe Beginning Was the Command Line (HarperCollins, 1999). Here's how he describes the difference between the hacker-built Unix and operating systems architected in the magazine mode:

The filesystems of Unix machines all have the same general structure. On your flimsy operating systems, you can create directories (folders) and give them names like Frodo or My Stuff and put them pretty much anywhere you like. But under Unix the highest level—the root—of the filesystem is always designated with the single character "/ " and it always contains the same set of top-level directories:

/usr
/etc
/var
/bin
/proc
/boot
/home
/root
/sbin
/dev
/lib
/tmp

And each of these directories typically has its own distinct structure of subdirectories. Note the obsessive use of abbreviations and avoidance of capital letters; this is a system invented by people to whom repetitive stress disorder is what black lung is to miners. Long names get worn down to three-letter nubbins, like stones smoothed by a river....It is this sort of acculturation that gives Unix hackers their confidence in the system, and the attitude of calm, unshakable, annoying superiority...Windows 95 and Mac OS are products, contrived by engineers in the service of specific companies. Unix, by contrast, is not so much a product as it is a painstakingly compiled oral history of the hacker subculture.

Is it any surprise that, in the long run (which we've been in since long before and after Neal wrote Command Line), hacker culture has pushed up, like lava through cracks in the Earth, so much useful stuff to build on?

Vernacular construction, with its valuing of craft over art, tends to produce what Stewart Brand calls "Low Road" buildings. These tend to be "low-visibility, low-rent, no-style." He adds, "Most of the world's work is done in Low Road buildings...and even in rich societies the most inventive creativity, especially youthful creativity, will be found in Low Road buildings, taking full advantage of the license to try things."

Not coincidentally, Stewart's ideal Low Road building is MIT's late Building 20 (http://www.eecs.mit.edu/building/20). Known as "The Magical Incubator," Building 20 was home to countless scientific advances: radar, microwave, spectroscopy, quantum mechanics, atomic and molecular beams, masers and lasers, atomic clocks, radio astronomy, linear particle acceleration, magnetron phasing, fiber optics, digital data transmission, and much more. More significant, for the purposes of the open source narrative, is what Stewart Brand reports here: "The Tech Model Railroad Club on the third floor, E Wing, was the source in the early 1960s of most of the first generation of computer `hackers' who set in motion a series of computer technology revolutions (still in progress)."

The term hacker is older than Moore's law, older than Unix, older than the whole software industry. Hacking has also persisted as a comparatively stable culture, while countless commercial "solutions" and fashions (moving to the top level there) have come and gone.

The construction industry makes a useful—even ideal—model for the software industry as it gradually matures because, while the software industry is a few decades old at best, construction is mature in the extreme. It might not be the oldest profession in the world, but it's probably the oldest industry. (Just ask the Masons.) It's also the largest industrial sector, with more than $3.5 trillion in revenues, worldwide. It understands and respects commodities (there's no fear and trembling about the "threat of commoditization"). It also finds plenty of ways to differentiate commodity products and make money with them. It understands the natural advantages and limits of patents and other intellectual property claims. It has lots of giant companies, but none that dominates like Microsoft does in personal computing, or like IBM used to do in mainframes. Nor does it have many (or any, as far as I know) large companies that make the 70-90% gross profit margins large companies in the software industry made in the days when there was no Net, and large vendors had to make their own infrastructure. (Or, in parlance that becomes more antique by the day, "be a platform provider".)

In fact, the ideals of modular and commoditized building materials have been around for a long time. Kim Polese, CEO of SpikeSource (disclosure: I'm an advisor to the company), is an industry veteran who labored toward the destinations of open source long before the means for getting there began to show up. "We tried a lot of good ideas," she says, "back when suppliers ruled the world: object-oriented programming. C++, CORBA, COM. Yet even our standards were isolated environments; no less monolithic than any vendor's silo. Our ideas—components, modularity, reusability—would have to wait for an open worldwide ecosystem to emerge."

So, we should stop and give credit where due to the vendors who worked in the meantime to build the proprietary infrastructures we call "platforms." Before the Net, and before a sufficient abundance of open source building materials appeared, there was often no choice. For some activities inside large enterprises, there still isn't much choice. If you're doing big-time Enterprise Resource Planning (ERP) or Business Process Management (BPM), there are no open source solutions out there. Still, the same used to be true of Customer Resource Management (CRM) and office private branch exchanges (PBXs)—to name two among many categories—but now that's changing with SugarCRM (http://www.sugarcrm.com/home) and Asterisk (http://www.asterisk.org).

So, the platform and silo system is still with us; it's just moving up the Civilization stack to the Commerce layer.

Smart software vendors who want to maintain their silos will still have to base them on free and open source infrastructure. That's what IBM is doing with Linux, which supports the company's proprietary DB2, Tivoli, and WebSphere products. It's what Apple did when it moved its whole silo from the decrepit Mac OS to Darwin, which is Appleized FreeBSD. And it's what Microsoft will do, eventually; but for now, we'll let that stand as prophesy.

The offerings are often mixed, so it gets hard (or annoying) to say what is open source and what is not. For example, while Apple contributes generously to the FreeBSD kernel (as well as to Apache, KDE, GNU, and other open source development communities), the OS X operating system Apple builds on BSD is highly proprietary. So is its popular iTunes software. In fact, iPods, for all their appeal, are essentially hardware extensions of iTunes software. They are a silo.

It would be a mistake, however, to dismiss Apple as a "proprietary" company. It is, but it also is not. Apple has an open source strategy. So do IBM, HP, Oracle, RealNetworks, Novell, Sun, SAP, and other large vendors that use open source strategies to support their proprietary offerings. All their strategies are different; but they are all based on an acceptance of open source as foundational infrastructure, on participation in open source development projects, and pm an appreciation for what open source provides to the world.

As long as we insist on treating open source and proprietary as polar opposites, we won't understand how complementary they can often be. Nor—if we are a company trying to succeed in a business world supported by open source—will we be able to come up with a useful understanding of how open source supports business, much less a strategy for putting that support to use.

For me, the wisest mind on this whole subject is Craig Burton. I first met Craig back in the 1980s, when he was one of the leading figures at Novell. As an old communications techie, I watched in amazement how thoroughly the networking vendors could screw up even a miraculously open standard like Ethernet. Even after IBM's PC became an industry standard, computer networking was a highly proprietary affair, in which customers faced baffling choices among local area networks (LANs) with names like DECnet, WangNet, OmniNet, Sytek, 3Com, Ungermann-Bass, Corvus, and IBM's Token Ring. There was thin and fat Ethernet, star and bus topologies, and a mess of noninteroperable standards and implementations, each based on what the trade called "pipes and protocols." Every vendor's set of pipes and protocols was a platform supporting a proprietary silo of products and services, sold exclusively to captive customers. Interoperability was a pie in the sky.

That whole paradigm was blown up when Novell, led by Craig Burton and Judith Clarke (she later married Craig and became Judith Burton), literally changed the networking conversation. When I first began to follow Novell, NetWare was a Motorola 68000-based file and print server. Craig brilliantly decided to move NetWare up a level, to become a Network Operating System (NOS) that worked independently both of hardware and of network pipes and protocols: essentially, as a networked filesystem for every PC, regardless of whose network connected those PCs. NetWare grew like wildfire. In the process, Craig and Novell reconceived networking as a set of services, and freed it from any notion of lower-level protocol and wiring dependency. When Craig, Judith, and Jamie Lewis later founded The Burton Group, their first achievement was formalizing the "network services model," which prevails to this day. When I said earlier that IM was not among the suite of standard Internet services, I was speaking in terms of that model.

Not long after I started with Linux Journal, I interviewed Craig a number of times, looking to get his point of view, both on open source and on the prospects for Linux (http://www.searls.com/burton_interview.html). He was greatly in favor of both, but he also had no patience with some of the rhetoric coming out of open source culture. For example, he took issue with the notion that open source and proprietary are opposites. "There are collapsed distinctions here," he told me. "The opposite of open is not proprietary, but closed. The opposite of proprietary is not open, but public domain." If you "uncollapse" those distinctions, he said, and lay them out orthogonally, you get The Burton Matrix.

Figure 15-2. The Burton Matrix

The Burton Matrix

Craig said that technical distinctions often collapse around moral sympathies. In the open source community, for example, "proprietary" and "closed" are considered bad, while "open" and "public domain" are considered good.

Figure 15-3. Collapsed distinctions

Collapsed distinctions

"However," he said, "if you remove morality as an issue, and spread out the two other distinctions that are collapsed, you have a good strategic framework for businesses to work with." To really take advantage of open source, he explained, you need to value ubiquity in your marketplace at least as much as you value scarcity in your product portfolio. In fact, your smartest move may be to take some of the products you're selling, and make them ubiquitous by moving them from proprietary/closed to open/public domain—literally, from scarcity to ubiquity.

Figure 15-4. Ubiquity creates infrastructure

Ubiquity creates infrastructure

This is a form of commoditization.

Figure 15-5. The Open Source strategy: commoditize

The Open Source strategy: commoditize

It's not the only one, of course. Most open source commodities are created from the start with the intention of putting them in the upper-right quadrant. But for businesses that want to create infrastructure and grow markets, this is one useful open source strategy. It's one of several a company can practice at the same time.

In IBM's case, the company adopted Linux (helping make it more ubiquitous), while also open sourcing Eclipse, moving it into the upper-right quadrant.

In Apple's case, the company open sourced nothing of its own, but used one of several other strategies: creation of a new standard (FireWire), adoption of an existing standard for the purpose of ubiquitizing it (USB, WiFi, ZeroConf/Rendezvous, MP3), and appropriation of an already developed codebase to save itself a lot of R&D work (FreeBSD, KHTML). Meanwhile, Apple has kept QuickTime and its growing portfolio of "i" applications on the proprietary side, even while opening them to free usage, essentially putting those in the upper-left quadrant.

When Tim O'Reilly had me show the Burton Matrix to Rob Glaser and Brian Behlendorf (Apache Foundation leader and founder/CTO of Collabnet), they helped me draw up the diagram shown in Figure 15-6 to explain how they were working together to open and ubiquitize many of the company's proprietary offerings (while also creating a development community).

Figure 15-6. Real's strategy

Real's strategy

Real's strategy, Rob said, was to move as much as possible from the lower to the upper left, and from the upper left to the upper right.

If we put the Burton Matrix beside the Long Now Foundation's "Layers of Time" model of Civilization, we see they have a common element: Infrastructure.

Figure 15-7. Time layers and the Burton Matrix

Time layers and the Burton Matrix

If we rock the Burton Matrix 45 degrees clockwise, superimpose it over the Civilization diagram, and cut out all but Infrastructure and Commerce, we have Fiure 15-8.

Figure 15-8. Matrix and layers merged

Matrix and layers merged

Now we can start to see how the world ought to work.

Figure 15-9. How it oughta work

How it oughta work

Open source Infrastructure supports Commerce while Commerce contributes to open source Infrastructure.

We may or may not like what IBM, HP, Google, Apple, Sun, Oracle, Novell, and Red Hat build on the open source Infrastructure that supports them, but we have to respect the fact that all those companies contribute back to that Infrastructure. There is another word for this form of contribution.

Figure 15-10. Commoditization is contribution

Commoditization is contribution

All of these examples, of course, pertain to technology companies. We don't see the same from Hollywood, which uses technology (including enormous quantities of Linux), but works against technology's interests, and with it the interests of the new world technology is busy creating.

Figure 15-11. How Hollywood sees it

How Hollywood sees it

Unfortunately, there are a few technology companies that feel the same way. The most notable among these is SCO. Once a leading Unix vendor, SCO transformed itself into the sworn enemy of Linux and open source, first when it sued IBM, and later when it sued DaimlerChrysler and AutoZone. Regardless of the merits of SCO's cases (I think they are groundless, but that's not what matters here), the Fear, Uncertainty, and Doubt (FUD) effects have been enormous and undeniable.

It has never been easy to get IT personnel to talk about what they are up to in any case. It's not their job to speak for the company (that's up to PR and marketing), and they often don't want their bosses, competitors, or suppliers to know what they're doing with anything, including open source. But still, some would talk—enough so that editors like me could pull together enough facts to tell a fair story.

That changed with the SCO lawsuits against DaimlerChrysler and AutoZone. These suits had a highly adrenalizing effect on the legal departments of large companies. Formerly open channels of communication became tightly closed legal sphincters. For example, two out the four panelists for my Do-It-Yourself IT (DIY-IT) discussion at the Open Source Business Conference (OSBC) in April 2004 were no-shows. One was no mystery: the speaker showed up, but told me he wasn't allowed to speak. Although he wasn't allowed to give the reason, his expressions made the matter clear: the company didn't want anybody saying anything in public about Linux or open source. I didn't learn what happened to the other panelist, Phil Moore of Morgan Stanley, until later in the year, when he rose out of the audience during a panel discussion at the O'Reilly Open Source Convention and spoke bluntly about what was actually going on. He began:

I work for the 38th largest company in the world, Morgan Stanley. We have a billion dollar IT budget. And we use a little of everything. Unfortunately. Excuse me, a LOT of everything. The trend I've seen in the last 10 years...is the exponential growth in the variety and the depth and breadth of installation of open source software in our infrastructure....What I'm seeing is that in the infrastructure, the core infrastructure, open source is going to take over, leaps and bounds....I'm predicting, right now, that by 2006 or 2007, we're going to be a 90% Linux shop.

At one point, Phil said, "We're still mostly a Solaris shop, but we are rapidly moving to Linux, though I'm not supposed to talk about that, for fear of being sued by SCO." Then he turned to Matt Asay (the Novell executive who ran OSBC) and added, "Which is the reason why I couldn't go to your conference, the OSBC. I wasn't allowed to go."

I ran those quotes, with Phil's permission, in my November 2004 column for Linux Journal. Phil no longer works at Morgan Stanley. He left voluntarily, but the fact that he's gone still speaks volumes. I want to thank him here for the honesty and courage it took for him to say what he did. Same goes for R0ml Lefkowitz (formerly) of AT&T Wireless, Roland Smith (formerly) of LSI Logic, Leon Chism of Orbitz, J.P. Rangaswami of Dresdner Kleinwort Wasserstein, and the rest of the handful of executives in large IT organizations who have talked to me factually and fearlessly about how open source—and Linux especially—are being put to good use in their companies.

Meanwhile, with a veil of silence over most IT departments, most of the publications covering IT are left with what comes easiest: writing about what vendors are doing. Or worse, treating open source projects as if they were vendors too. "Microsoft v. Linux: Who Will Win?" the headlines say. Again and again.

It's a false fight, and it always has been. We're making a world here. There's a limit to how well you can live in it and still ignore the fact that it exists. And how it got here.

Notes

  1. I had the same kind of trouble when I first started hearing everything one could communicate referred to as "content." I was a writer for most of my adult life, and suddenly I was a "content" provider. This seemed ludicrous to me. No writer was ever motivated by the thought that they were "producing content." Their products were articles, books, essays, columns, or (if we needed to be a bit more general), editorial. "I didn't start hearing about `content' until the container business felt threatened," John Perry Barlow said.
Personal tools