Open Sources 2.0/Beyond Open Source: Collaboration and Community/Communicating Many to Many

From WikiContent

Jump to: navigation, search
Open Sources 2.0

Jeff Bates and Mark Stone

Typical users in the Windows community and typical users in the Linux/open source community have different tendencies. For real-time communications, Windows users tend to prefer an Instant Messaging (IM) client like AOL Instant Messenger (AIM) and Linux users tend to prefer an Internet Relay Chat (IRC) client like XChat. On the surface, these clients and protocols differ little: both support channel or chat room messaging, both support one-on-one messaging, and both allow for some degree of moderated discussion. Yet IM users tend to communicate one-on-one by default, resorting to chat room discussions only for specific purposes and even then rarely, and IRC users tend to communicate in group channels, resorting to one-on-one communication only occasionally.

That tendency toward group activity underlies much of the collaborative instinct in the open source community, and ultimately provides a key to open source's remarkable success. Yet any IRC veteran knows well the scaling problems group communication encounters. A channel with a dozen or so participants, a handful of whom are vocal, can be a very productive center of communication. A channel with 20 to 50 participants suffers a crippling signal-to-noise ratio, absent some form of moderation: too much noise, not enough signal.

This pattern is reflected in all forms of online group communication: discussion forums, email lists, and Usenet from the very early days. Breaking this pattern stands as one of the chief challenges to effective collaboration. How can communicating many to many scale? What conditions enable network effects to take hold so that more participants improves rather than diminishes the quality of communication, and hence the power of collaboration?

Slashdot stands as a striking counterexample to the usual pattern. Over the years, the site has evolved into a high-quality, moderated discussion forum, one where the more people participate, the more valuable the discussion is. Network effects have taken hold; many-to-many communication is enabling a unique form of collaboration.

Contents

The Origins of Slashdot

Image:Open Sources 2.0_I_9_tt29.png


Like many web endeavors that evolved into dot-coms, Slashdot started as a hobby and a learning exercise. Rob Malda, Jeff Bates, Nate Oostendorp, and Kurt DeMaagd—the future "Blockstackers"—had been friends growing up in Holland, Michigan, and later at Hope College. At first, "online community" meant bulletin board systems (BBSs): email, discussion, and file sharing services available locally by dialing up and connecting directly to a BBS machine. College provided access to the Internet and the earliest web sites via one of the college's Unix or VAX machines.

Rob Malda's first home page at college (http://cs.hope.edu/~malda), dubbed "Chips and Dips," would be labeled by mainstream media today as a weblog, or blog. Superficially, Chips and Dips did resemble current blogs minus many of the interactive elements we take for granted in today's Internet. It was a personal page for Rob. It offered his opinions on everything from web design to science fiction book reviews. Yet to call it an early blog really misses the point.

Blogs are fundamentally inward facing. They share events from the blogger's life, together with opinions of the blogger on those events and the world at large. Chips and Dips had none of that intimate and voyeuristic sense of a diary. Instead, it had more in common with the early versions of Yahoo!: a hand-built directory of useful links, with guidance and commentary, aimed at like-minded people. Chips and Dips was, from the beginning, about that sense of community; it was outward facing.

Indeed, right from the start Chips and Dips offered more than just Rob's links and opinions. Friends submitted suggested entries for the directory listings, or reviews of movies that others had yet to see. The site was just flat HTML, no CGI or other dynamic elements. As a consequence, the only way to submit to the site or contribute to discussion was to email Rob, and then wait for Rob to post on the site. This created an implicit and autocratic moderation mechanism. If Rob did not have the time or interest to post something, it didn't get posted. As it was Rob's site, his decision, or even whim, was final.

In the fall semester of 1997, Rob Malda, Jeff Bates, and the other Blockstackers entered their junior year of college. Most had two years of computer science under their belts, and lots of hands-on experience from hobbies pursued and an assortment of student jobs. They also had a sense of the larger world of which Hope College was a part.

Netscape had completed its successful IPO. Graphical web browsers, and the Web itself, were becoming pervasive parts of popular culture. Microsoft had released Windows 95 and announced that the Internet was the future of the company. Linux was headed toward the 2.0 release of the kernel. Apache was, as it is today, running most public web sites. The dot-com boom was in full swing.

Registering a domain name had gone from an esoteric to a more commonplace activity, albeit an expensive one by student standards. While Rob had done a lot of computer programming, he was fundamentally a designer, and indeed one with a strong sense of the ironic. He approached problems visually, and in his spare time was as likely to be doodling cartoons as writing code.

The choice of the name "Slashdot" for a domain was a clever play on the line between the visual and the verbal. In the early web days, the idea of a URL and what it was still seemed alien to the mass media. Every ad for a web site began with the announcer carefully spelling out, "H-T-T-P colon slash slash..." Visually, "/" is simple and distinctive. In those days, verbally spelling out "H-T-T-P colon slash slash....dot org" was ridiculous. It appealed to Rob's sense of humor.

Moving from Chips and Dips to Slashdot brought several immediate changes, not all of them forseen.

Slashdot in the Early Days

Image:Open Sources 2.0_I_9_tt30.png


From fall of 1997 to spring of 1998, Rob Malda and the other Blockstackers went through a rapid education on emerging web technologies. Chips and Dips had been entirely static HTML. By spring of 1998, Slashdot had dynamic content through CGI, and then very quickly, given the performance limitations of CGI, dynamic content via Apache modules, specifically mod_perl. The quality of HTML improved, coming much closer to standards compliant.

The purpose at this stage was a learning exercise as much as anything else. New features of HTML or Perl were learned during "day jobs" at work, and then what was learned was applied to Slashdot. By the spring of 1998, this process had begun to reverse itself. New ideas were tested on Slashdot, often proving valuable at day jobs as well.

One of the important changes during this time was the creation of submit.pl, the Slashdot submissions bin. Making submissions an inherent part of the site rather than something that had to pass through the Inbox of Rob's email made the submissions process scalable, and also enabled others besides Rob to take a direct hand in the editorial process of reviewing submissions and approving them for posting to the site.

Another key technical change during this time was the move from a web site that was merely under an account on a university server to a standalone server running Slashdot. The hardware was a single DEC Multia that Rob had received as barter payment for drawing cartoons for a self-published business book. The server was hosted at Rob and Jeff's place of employment. Their employers needed an email server, and Rob suggested that they could use that box as an email server, as long as he could do some other hosting on the box as well. Initially this arrangement worked well, though the server was subject to sudden power outages and downtime if you weren't careful to avoid the power cord when putting you feet under Rob's desk. Of course, it was a server running Linux.

While Rob's interest in Linux dates back to Chips and Dips, running Slashdot and the technology behind it really increased that interest. Naturally, content on the site itself served this need, as Rob began to accumulate, and visitors continued to send in, an impressive list of Linux resources online.

This dynamic of needing Linux to run the site and using the site to learn about Linux created an unanticipated side effect, compounded by other changes happening at the same time. Chips and Dips, while it was Rob's personal home page, had always been a community site. Indeed, this community focus is one of the key differentiators between Chips and Dips and what today we would call a blog. However, that initial community consisted primarily of other geeks in and around Holland, Michigan, and specifically, others at Hope College (though even early on the larger Linux community had interest in some of Rob's graphics work).

A university home page and a top-level domain like "Slashdot.org" are, in some sense, equally public. Both could be accessed from any browser connected to the Web anywhere in the world. But Slashdot felt like a more prominent site than Chips and Dips, and was more likely to be bookmarked by others, or added to directories, which, at the time, were still largely compiled by hand. This alone drew a larger, more global audience to Slashdot.

In addition, in early 1998, there simply weren't that many thorough Linux resources online. As one of these few, Slashdot stood out to Linux enthusiasts everywhere, not just at Hope College. The site was still a community site. But somewhere along the way in winter of 1997-1998, the community had become global. This was still a community of like-minded peers; it was, as Slashdot has proclaimed from the early days, "News for Nerds." But it was becoming a much larger community. During this period, Slashdot passed 20,000 page views per day, a level of traffic that, especially circa 1998, signified a sizable, loyal audience beyond the confines of Holland, Michigan. By way of comparison, Holland has a population of a little over 30,000.

Some years later, then-Slashdot columnist Jon Katz speculated about why this important resource for the technically inclined had happened in rural Michigan rather than in a flourishing center of technology like Silicon Valley. Katz felt that it was a community born of necessity. In Silicon Valley, technical communities abounded, and face-to-face opportunities to meet and interact with like-minded peers were plentiful. Only in a more isolated place like Holland, Michigan, would it be necessary to actively pursue a community online to find the critical mass needed to form a genuine community of interest.

A community of like-minded peers is one thing. The arrival of the masses is quite another. The Slashdot community was about to change dramatically, starting with what seemed like a small incident.

The Slashdot Effect

Image:Open Sources 2.0_I_9_tt31.png


On January 12, 1998, Rob Malda posted a piece on Slashdot titled "Simple Solutions" that he had written, and that he described as "the first of hopefully many Slashdot.org Editorials" (see Appendix D). In this editorial, he challenged Netscape to open source its browser code as the best available alternative for a company losing both money and market share. On January 22, Netscape put out a press release with the headline "Netscape Announces Plans to Make Next-Generation Communicator Source Code Available Free on the Net." The release of Mozilla marks one of the signature events in open source development.

It's important to understand what did not happen. No one at either Slashdot or Netscape has ever claimed that Rob Malda's essay had any direct effect on Netscape's decision. Indeed, the complexity of the process and the proximity of the dates suggest that Netscape's decision must have been made well in advance of Rob's editorial. But the proximity of the two events—and a general lack of understanding about open source at the time—caused mainstream technology media to link the two.

Slashdot had been discovered by mass technology media.

Several changes began to take hold on Slashdot, and indeed, the effects of those changes are still playing out today. The mainstream technology media took a regular interest in Slashdot, and the mere fact that a story was covered on Slashdot became significant. Ironically, Slashdot seldom has been the originator of a news story, so this interest in Slashdot coverage was largely about watching what other people were watching, an obsession that seems to be a distinctive part of the Internet generation.

A peculiar side effect of this "watching the watchers" was that Slashdot became a source of journalistic research. Smart journalists looked for insightful comments, finding stories and ideas in those comments and their authors. Indeed, because Slashdot has always permitted anonymous posting, comments often had the insider's candor that journalists value.

An extension of this media interest was the involvement of media figures in Slashdot. Author Jon Katz was working at HotWired at the time, while also researching his book, Geeks. He contacted Rob and Jeff first as part of his research for the book, but more and more because of his genuine interest in Slashdot and the community for which it stood. The result was that Katz became a regular columnist on Slashdot for several years.

The reaction to Katz's presence was revealing. Many were impressed with his writings and insights into geek culture, and would quietly send their notes of appreciation to Rob, Jeff, or Jon privately. At the same time, a vocal minority of the audience objected strongly to Katz as an outsider, and posted their views bluntly as comments to any column he posted. When user accounts and customizations arrived on Slashdot, "filter out Jon Katz" was, for a time, the most frequently selected customization.

This response was, of course, out of all proportion to anything Katz said or did. Katz is a professional writer, a serious journalist, and had a genuine interest in the geek community. However, he became a symbol of the new crowd that had arrived at Slashdot, readers who were more interested in geek culture than geek technology. Katz became a lightening rod for all the resentment felt by the original core audience, some of whom felt the need to lash out at these "invaders."

Slashdot was reaching traffic levels that signified a vastly larger audience than the original Linux and open source enthusiasts and other geeks. On February 9, 1998, the site received its 1 millionth hit. Barely a month later, on March 18, it received its 2 millionth hit. Just a month after that, the site recorded more than 100,000 page views per day.

Yet if Slashdot was changing, it was also changing other sites as well. The appearance of a particularly noteworthy story at the top of Slashdot's home page would generate a flurry of discussion. Few stories posted left the front page with fewer than 200 comments, and many stories received in excess of 700 comments. Further, much of the Slashdot audience would descend, simultaneously and en masse, on the site from which the story originated. Many sites were unable to handle this sudden influx of new traffic, and would simply crash under the load.

This sequence of events—the posting of a story, the rush of traffic to the story's site, and the strain or failure of the site under the load—has become so notorious that it is now known as "the Slashdot Effect." Indeed the network characteristics of the Slashdot Effect have been worthy of academic study. Stephen Adler at Brookhaven National Laboratory had the first real study of the Slashdot Effect, available online at http://ssadler.phy.bnl.gov/adler/SDE/SlashDotEffect.html.

The term Slashdot Effect has entered popular culture; it has an entry in the Oxford English Dictionary's online edition, and Slashdot serves as the answer to a question in the '90s edition of Trivial Pursuit.

Trolls, Anonymous Cowards, and Insensitive Clods

Image:Open Sources 2.0_I_9_tt32.png


Many discussion forums, be they mailing lists, web-based discussions, or IRC channels, produce high-quality discussions among a few participants. The challenge is to scale this "few-to-few" communication all the way to "many to many." Adding a moderator helps to a point. Many moderated discussions, though, improve the "signal-to-noise ratio" at a cost: over time, the signal takes on more and more of the moderator's particular viewpoint. Other viewpoints, intentionally or not, are stifled with a hostile reaction. Rarely can a discussion forum have enough commonality in interest to draw a critical mass of audience, have enough variety in viewpoints to keep the discussion interesting, and scale to ever-larger audience sizes without losing the signal of the discussion in the noise of the chatter.

Discussion systems can avoid the tyranny of a single moderator by letting all users vote on moderation. However, letting every user vote equally on moderation assumes that all users are equally informed, equally concerned, and equally motivated about what they are moderating. These are unrealistic assumptions that lead to a tyranny of the masses as damning as the tyranny of a single moderator.

In the spring and summer of 1998, Slashdot faced all these challenges. What emerged was a mix of software engineering and social engineering to cope with the growth of the site. Not everything that was tried worked, and some solutions are still works in progress.

User accounts were created. Registered users could customize which slashboxes (sets of links to changing external content; a precursor to today's RSS) appeared on their Slashdot page, and where. Today we would call this portalizing the site. To the Slashdot team it was a way of minimizing complaints and encouraging visitors to register and log in. Users could filter content to see only stories on certain topics, or to exclude stories on certain topics. Having a low user account number also became a point of pride with the regular visitors.

An interesting and intended side effect of user accounts was to reduce the number of off-topic or deliberately inflammatory comments. People are, by nature, less inhibited when communicating anonymously. The simple step of encouraging people to identify themselves helped restore some order to discussions.

An issue continuously debated behind the scenes was whether anonymous posting should be allowed at all. In the end, free-speech considerations have trumped all other considerations in this debate. Anonymous posting allows an employee to speak out candidly about his employer without fear of retribution. Anonymous posting allows someone to express an opinion or political view without being chastised by their peers. Ultimately, anonymous posting contributed significantly to key events that have left an indelible mark on Slashdot.

One step that also occurred was the slowly diminishing value of anonymous comments—over time, their base score became set to 0, which was lower than the 1 threshold that the non-logged-in user reads at by default. This means that most of the people only read anonymous posts that have been moderated up.

As the audience grew, so did the number of submissions. It became impractical to get every worthy submission posted to the front page; stories simply would have scrolled by too fast. In May of 1998, sections were added to Slashdot, creating a separate front page for those interested in a specific topic. Today there are 14 sections on Slashdot, ranging from Book Reviews, to Science, to Politics.

The most controversial modification to Slashdot came in October 1998, with the introduction of moderation. Each comment is classified on two dimensions: one for the type of comment, and the other for the quality of the comment. Types of comments include: Overated, Underated, Troll, Insightful, Informative, Redundant, Offtopic, and Flamebait. Quality is numeric, from -1 to 5. All comments initially started at mod level 1, but could be moderated up or down for as long as the discussion on that story was open.

One of the important settings available to registered users is the moderation level. Users can select what level of comments they see by default, and can change that setting on a story-by-story basis. Thus, if the number of comments at a certain level is too many to read through, or the relevance drops off too much, users can filter out comments below a certain level. The default moderation level also enables a form of filtering without censoring. The most irrelevant or inflammatory comments routinely get moderated down to a level of 0 or -1. An unregistered user visiting the site has his default moderation level set to 1. Since 95% of all visitors to the site never change their moderation level, the vast majority of visitors never see the lowest moderated comments. Moderation also provided an additional motivation to get users to register. After a time, the policy was changed so that an anonymous user's posts generally started with a moderation level of 0, but a regsitered user's comments started with a moderation level of 1.

Many discussion forums have tried various forms of moderation. The challenge is to find a system that is scalable—Slashdot routinely generates tens of thousands of comments per day—and heterogeneous, namely reflecting more than a single point of view about how comments should be moderated.

Slashdot met these challenges by borrowing from the principles of collaboration in its open source roots. The audience is essentially self-moderating, and indeed the more people participate in the system, the better the moderation gets. This is the enormous differentiator for Slashdot. Where most discussion forums crumble under a deteriorating signal-to-noise ratio as their size increases, Slashdot actually benefits from network effects: more is better.

The key is that Slashdot tracks a wide range of information about its users: how many comments a user has posted, how many stories a user has submitted, how many submissions have been accepted, what moderation level a user's comments tend to settle on, what type of comments a user typically makes. All of this data is combined to produce a number that roughly quantifies the value of a user to the site. This number is referred to in the Slashdot system as karma.

Users with high karma are periodically selected to moderate. The system is automated, requiring little intervention from the Slashdot staff. Once selected, a user has his moderation authority turned on for a period of time, enabling him to moderate up or down, or classify comments he reads. After a period of time, moderation is turned off, and passes to another user. At any given time there are roughly 1,850 users moderating comments.[1]

Originally, a user's karma number was viewable. This policy led to problems. Users viewed their karma rating as a score, and raising their karma as a game. Once people tried to deliberately game the system, the whole system no longer functioned as well.

The Slashdot staff was also inundated with email complaints about karma—for instance, "My latest comment was modded up to 5 but my karma went down; I think your system is broken." Of course, in this context, karma is just a technical term for the sum of a formula used in the Slashdot system; as such, it could not possibly be "broken." Furthermore, the code for Slashdot has always been available as open source, meaning that those who really wanted to understand their karma rating could have done so. Human nature being what it is, however, people quickly slipped into thinking that there was some real thing to which karma corresponded and which the system was trying to approximate. In the end, the only workable solution has been to keep karma ratings private and give users only a vague approximation of their karma ratings.

Slashdot also evolved to have a number of "social engineering" elements that did as much to channel users' behavior as any of the technical features. Some of these social engineering elements are blatant, like referring to not logged-in posters as "anonymous cowards." Similarly, wildly off-topic or deliberately inflammatory posters are referred to as "trolls."[2]

Some of these elements are subtler. Watch a first-time visitor try to find the "submit" link, for example. Visitors aren't actively encouraged to submit; you have to really want to get your submission in. Creating that barrier to entry means that the overall submission is of higher quality. Because some effort is required to learn how to submit, people who put some thought into their submissions are more likely to submit.

The site also has a distinct personality, one that does not take itself too seriously. This is apparent from the self-deprecating tag line ("News for nerds; stuff that matters") to the obvious humor in many of the weekly polls. Staff on the site are referred to by their nicknames, usually originating as nicks on an IRC network or handles on a BBS. These nicknames have deliberate cultural references meant to be understood by an audience with the right cultural background. "Cowboy Neal" derives from a character in Kerouac. "CmdrTaco" is a reference to a Dave Barry column. If you haven't watched "The Simpsons" or "South Park" regularly, much of the humor on Slashdot will pass you by.

In its own way, this too is part of the social engineering. The site personality and the cultural references are a subtle test for like-mindedness with the audience, a way of encouraging participation from those who "get it" and distancing those who don't.

Slashdot grew up a lot as a site in the summer of 1998. That spring the site began running banner advertising as a means of generating revenue. Banner sales were originally outsourced to a third party, but out of frustration with the ad sales company's inability to manage sales or understand the Slashdot audience (who still vociferously complain at the use of Flash animations in ads), advertising sales were brought back in-house in July of 1998 and were managed by Jeff Bates.

With little fanfare, Rob Malda quit his job in August 1998 and became Slashdot's first full-time employee. While other staff continued to work on a part-time or volunteer basis, this was a significant milestone. Less than a year after the registration of the Slashdot domain name, the Blockstackers had gone from running a web site to running a business.

Columbine

Image:Open Sources 2.0_I_9_tt33.png


Slashdot has moved away from its technology roots only gradually, and only where a new topic clearly connects to its core audience. There is a "Book Review" section, but its focus is on computer books. Sections like "Your Rights Online" branch farther afield, but are still rooted in topics of concern to the technically inclined: the SCO lawsuit, the legality of file sharing and peer-to-peer networks, or the status of the Digital Millenium Copyright Act (DMCA).

Slashdot has never tried to be a general news site, and events of general interest typically are not covered. In this regard, what arrives in the Slashdot submission bin and what is published to the front page differ substantially.

On Wednesday, April 21, 1999, a number of Slashdot readers submitted reports of a shooting at Columbine High School in Littleton, Colorado. Ultimately, it would emerge that fifteen people died, including the two teenage boys who were the shooters. Twenty-three other people were injured before the two boys ended the massacre by committing suicide. These details were far from clear as the first submissions arrived at Slashdot that Wednesday.

A breaking story like Columbine brings out the best and worst in Internet news reporting. The Internet has an unprecedented capacity to cover events in real time, and to draw from a widely distributed network of sources. At the same time, standards for journalism and fact checking are largely undefined in this new medium.Rumor is reported as fact, and the distributed nature of information flow makes correction in light of new information difficult.

When the Slashdot staff looked at the submissions regarding Columbine that Wednesday, it seemed clear that first of all, the story did not tie directly to the technology topics that are the core of Slashdot's coverage, and second, that the story was still an emerging one with important facts still unknown. Without much discussion, the staff decided not to post the story, and leave coverage of Columbine to the mainstream media.

Columnist Jon Katz, however, saw a different story emerging over the next two days. Katz had been hard at work on his book Geeks, and indeed it was this research into geek culture that had drawn him to Slashdot in the first place. An important part of that research concerned the isolation and alienation felt by geek teenagers simply because they were smart and different. Katz was appalled by the quick move of mainstream media to stereotype the Columbine shooters as a deranged byproduct of a violence-desensitizing subculture of hard rock, computers, and video games.

On Friday, April 23, Katz used his column on Slashdot to voice a response. He published a piece titled "Why Kids Kill." Katz's main point was to counter the stereotype. He argued that youth violence was dramatically on the decline, and that there was no research to establish a correlation between violence in movies, music, games, or television and violent behavior among youth. He also raised the question of why our expectations of who violent youth would be—urban, disadvantaged, and ethnic—do not match the reality of who violent youth actually are—suburban, middle class, and white. The suggestion, though subtly stated by Katz, was that stereotypes provided a convenient alternative to confronting the fact that today's parents do not understand today's youth. If technology was responsible for anything, it was for widening that gap in understanding.

What followed was unprecedented in the history of Slashdot. Most Slashdot readers view the site and comment from somewhere other than a home computer. Either they are at work, or they are students using a university computer. Consequently, traffic on the site, and number of comments, tend to decline on the weekend, with a big drop-off starting on Friday. Katz's column was posted at 11:00 A.M. on Friday. It received over 1,000 comments (see http://slashdot.org/article.pl?sid=99/04/22/2136230&mode=thread).

The comments included some from parents, teachers, and other adults. But the vast majority of comments were from teenagers. They spoke out not to defend the shootings at Columbine, but to express their own feelings of alienation. Much of this alienation was rooted in the struggle to grow up in a world of rapid, technology-driven change to which the adults in their life could not relate. The comments are heartfelt, surprisingly articulate, and seemingly countless (footnote: the "countless" part is, in some sense, true. Slashdot does not archive comments moderated below zero, so while we know that more than 1,000 comments moderated zero and higher, there is no record of the total number of comments posted).

It's worth quoting a representative example:

When I was growing up, I wore a lot of black, I studied explosives and bomb-making, I learned how to shoot, and I memorized complete copies of _Jane's Infantry Weapons_ and various army and special forces survival manuals. It was a funky hobby that never really went anywhere. I've worn a black trenchcoat almost every day for ten years, I've played DOOM-like games since they first appeared, and I'm a big fan of John Woo films. To the best of my knowledge, I never went nuts and killed anyone.I also graduated at the top of my high school class and graduated with honors from an ivy-league college, and I'm now happily married and managing the support team for a successful tech startup. I give credit for all of my success to my parents, who took an active interest in what I was doing and why, without trying to control my life.

Katz and the others at Slashdot were stunned. While the site didn't crash, the volume of comments put the site under an unprecedented load. Katz's personal email was flooded with messages from young people contacting him directly to tell their own stories of alienation and ostracism. Everyone at Slashdot realized that they had tapped into a deep sentiment in urgent need of expression.

On Monday, Katz posted a new column, "Voices from the Hellmouth" (see http://slashdot.org/articles/99/04/25/1438249.shtml). Katz could have tried to assert himself and lead the discussion at this point. He chose not to. Instead, he recognized a stillpent-up need for the discussion to continue, and recognized that the most useful thing he could do was facilitate, rather than lead the discussion. His Monday column contained very little of his own words or opinion, and was instead his attempt to relay the most insightful or poignant stories he had received. More than 1,200 comments were posted in response to "Voices from the Hellmouth."[3]

Monday's comments continued the themes of Friday. Young people expressed how frustrated they were that parents and teachers disapproved of their interests, their community, and their culture simply because it was something adults did not understand. Young people expressed how isolated they felt when teased and persecuted by their peers for dressing different, acting different, and worst of all, being smart.

On Tuesday, sensing that the discussion had not yet run its course, Katz posted another column, titled "More Stories from the Hellmouth" (see http://slashdot.org/features/99/04/27/0310247.shtml). More than 500 comments were posted in response. It's interesting to note an update that Rob Malda inserted into the story late in the day (around 7:45 that evening): "Sharon Isaak from Dateline NBC wants to get in touch with folks to do a story on this subject for this show. She's specifically seeking Jay of the Southeast, Anika78 of suburban Chicago, ZBird of New Jersey, Dan in Boise, Idaho, but she'd also like anyone who's been targeted as a result of this thing to contact her. Wonder if they make ya wear pancake makeup..."

The Columbine story was now a week old.

Two facts about Slashdot are easy to overlook in the course of more-routine day-to-day content that appears on the site. First, Slashdot is a discussion site, not a news site. Breaking stories are seldom reported on Slashdot, and while a great deal of news coverage is presented on the site, fundamentally the purpose of the news is to seed discussion. Second, the audience of Slashdot is not so much an audience as it is a community.

Before Columbine, even Slashdot's regulars may not have realized the extent to which they were a community. Rob Malda and the other Blockstackers had elevated themselves from an isolated community of geeks in Holland, Michigan, to a global community of like-minded peers. The "Hellmouth" series on Slashdot affirmed that it was not just the staff of the site that could make this transformation, but the audience as well. Over the course of that post-Columbine week, teenage geeks became the voice of Slashdot, speaking many to many. They recognized and celebrated that they were not alone, but were part of a larger community. The site, like Katz, receded into the background, and communication was direct between those who came to the site and posted. Nor was the discussion that week merely a collective pat on the back. Practical, meaningful advice was asked for, offered, and shared: crisis centers to contact, teachers to recommend who had been particularly understanding, programs and opportunities that catered to the aspiring geek.

That practical dynamic is an essential ingredient of community, and had been a characteristic of Slashdot for some time. Those who think of the site merely as a news site have overlooked a small but important section of the site called "Ask Slashdot." Debuting in May of 1998, this category of posts was not affiliated with any seeding news story, but instead was a direct plea for advice from the community.

At times the "Ask Slashdot" posts have looked suspiciously like questions that Rob or other staff members would like to have answered: how to set up a local wireless network, or wire a home theater system. This is part of the meaning of like-minded peers, however; questions of interest to one member of the community are usually of interest to many others as well.

Over the years, the archives of "Ask Slashdot" have grown to an impressive respository of advice and how-to information for those immersed in the geek lifestyle. More than any other section, "Ask Slashdot" exemplifies the community aspect of the site, with the audience speaking directly to each other, unfettered by any lead-in story.

The week following Columbine exemplified the finest characteristics of that community spirit.

Slashdot Grows Up

Image:Open Sources 2.0_I_9_tt34.png


If the Netscape story had introduced Slashdot to mainstream technical media, Columbine introduced Slashdot to the mass media. The site's audience grew, and diversified. The attention focused on the site grew as well. With Columbine, the media recognized that Slashdot could be, on occasion, not just a discussion point for news, but the news story itself.

Comments on Slashdot have become increasingly sophisticated. View Slashdot comments at moderation level 5 and you see a number of lengthy and thoughtful commentaries on each story. You also see certain usernames recur as authors of particularly insightful comments. That combination of an easy online vehicle for expressing opinion and building reputation through regular posting of well-considered opinions has made Slashdot a precursor for the blogging movement of today.

Story submissions have also become more sophisticated. In the first couple of years, Slashdot accepted many submissions that were simply a link to an interesting bit of technology news and a brief description of the news item. Accepted submissions now typically have a main story link, one or more background links to other related stories, as well as links to past Slashdot discussions on the same topic, all contained within a paragraph or more of explanation. This richer form of submission is a result of one of the network effects behind the site. The site is now large enough that any important story will be submitted multiple times, enabling the site staff to pick the most complete and well-formed submission for actual publication to the front page of the site. This puts the regular submitters in tacit competition with each other to create the best submission for key stories.

The continued success and popularity of the site owe as much to its anonymous contributors as to its regular, registered users. Slashdot could not have been the singular "town hall" that it was in the wake of Columbine without allowing and supporting anonymous posting. That aspect of the site has become ever more complicated to manage and maintain, however.

There are legal threats to anonymous status. While Slashdot has yet to be asked to turn over its logfiles, legislation from the DMCA to the Patriot Act has the potential to force Slashdot's hand on this issue. Furthermore, the Motion Picture Association of America (MPAA) and the Recording Industry Association of America (RIAA) have made it clear that they will use civil action wherever they think they have a chance of uncovering the identity of supposed copyright violators.

One change at Slashdot in response to this new legal climate has been to handle logfiles differently. Most sites keep logfiles of daily visitors to the site, and differentiate visitors by Internet Protocol (IP) number (the numerical address computers use to identify themselves to each other over the Internet). This IP number can potentially be used to backtrace where a particular visitor to a site came from. Slashdot now scrambles the IP numbers in its logfiles using a strong encryption scheme, and then discards the encrypting key. The result is a unique encrypted number associated with each unique IP number, but no way, even by the Slashdot staff, to derive the original IP number from the encrypted number stored in the logfiles.

As much of a problem as external challenges to anonymity is the mischievous behavior of a few anonymous posters. Given the high profile of Slashdot, there is prestige in the "troll" community associated with defacing or bringing down the site.

Many attacks on the site attempt some form of exploit on the comment system. These can include proxy flooding (repeated comment submissions dispersed via different sources using open proxies; similar to a "denial-of-service" attack); comment binging (attempting to overload the system with large comment submissions); and script attacks (using a script to generate a nonsense comment that can be submitted repeatedly at high speed).

Slashdot now has a maximum comment size, as well as a test for and block of open proxies. It has an internal definition of a well-formed comment, and roughly 500 separate regular expression tests, written in Perl, to which each comment submission is subjected. The attacks continue, but anonymous posting has never been disabled, and remains a bedrock principle of the site.

Sheer scaling issues have presented a different challenge. As of this writing, the site delivers 3.9 million page views per day to 400,000 unique visitors.[4] While Slashdot's initial moderation system worked well for a while, moderation has had to become more sophisticated to both handle the larger volume of submissions and comments and take better advantage of the network effects inherent in an audience of this size. The most notable change has been the introduction of metamoderation. In metamoderation, select users are asked to moderate the moderators. The metamoderators review both comments and moderation decisions about those comments, and respond with a simple "agree, disagree, or no comment" response. Those selected for metamoderation typically have about 20 moderation decisions to review when metamoderation is turned on, and then might not metamoderate again for several weeks or months. The results give the staff and the Slashdot system a more fine-grained picture of which regular users of the site are effective at moderation, and which are consistently contrarian.

One of the lessons of Columbine was that the site not only had to be restructured to meet regular, steady growth in traffic, but also had to be capable of responding to surges in traffic associated with an extrordinary news event. While the sequence of events around Columbine never brought the site down, the staff realized they had, in many ways, been fortunate. They had not covered Columbine the day of the event. Katz's first story had been posted on a Friday, a low-traffic day. They had a weekend to recognize the effects of his story and anticipate the follow-up. Overall they had a whole week to work through the process. In many ways, Columbine was the exception; it is unusual for a news event to play out that gradually.

September 11

Image:Open Sources 2.0_I_9_tt35.png


Department of Defense money originally funded the research that became the architecture of the Internet. That research mandate was based on the perceived needs of a late-'60s Cold War nation. One of the design constraints was this: in the event of a selective, possibly nuclear attack on the United States, could a data network be designed that would continue to function despite outages to significant grid sections of the network? In other words, could the architecture of the network allow for graceful degradation and an opportunity to route around outages?

Two principles that underly the Internet architecture are decentralization and redundancy. Think, for example, about the way in which the Domain Name System (DNS) is implemented. This is the protocol by which a computer knows how to associate a human-readable address such as Slashdot.org with a computer-readable IP number. DNS has no single, canonical server to provide an authoritative list of these mappings. Such an approach would create a single point of failure that would serve as a bottleneck under high-traffic conditions and would bring down all DNS-dependent traffic in the event of a server failure.

Instead, DNS is more of a peer-to-peer network, with thousands of servers across the Internet functioning as DNS servers. Any server can update its records, and its updates will gradually be propagated to other DNS servers. In the course of a couple of days, any change to one DNS server can reach all others. Nor does one need "permission" to put up a DNS server. The requisite software is open source, and the Internet architecture is designed to automatically accept new servers or respond when encountering missing or offline servers. The system is highly distributed and redundant.

Even the basic network rules about how packets are routed from one destination to another follow these principles. Any computer sends a "test packet" first, attempting to establish a route to its destination. Once a route is established, actual data packets are sent. If at any point, the originating computer fails to get an authenticating response for that route, it explores for a new route and continues sending packets along the new route. There are no canonical, authoritative routes from Point A to Point B; each network route is a process of discovery based on current network conditions.

The principles are simple enough: avoid single points of failure by relying on a highly distributed network of peers rather than one or a few hubs around central, authoritative servers. The network protocols that employ these principles, however, are only as robust as the applications that use them. All of that redundancy and flexibility in routing does no good once email is queued up at an unresponsive destination server. If millions of requests are all headed for the same web server, that becomes the de facto center of an unresponsive hub.

In other words, to benefit from the design features of the Internet architecture, an application must be specifically tailored to use those features. In fact, relatively few applications do make use of this underlying structure. One application that does is IRC, the staple of online communication in the open source community.

A look at the network list in a default setup of XChat (a common IRC client) reveals dozens of IRC networks. Some are based around a common interest, like QuakeNet; some are based around a geographical location, like OzNet. Many, like Freenode, are general purpose. Within each of these networks will be dozens, or even hundreds of channels, each of which represents a particular community or topic of interest. The more popular networks easily have tens of thousands of users connected simultaneously at any given time.

IRC puts very slight demands on a server; all of the transmissions are short strings of text. Many universities and a large number of commercial sites volunteer server space to run an IRC server. All of the servers that are part of a given network work together to mirror the activity on the network. Typically servers in a network are partitioned into groups, with each group responsibile for mirroring a subset of the channels on that network. In the event that a given server goes offline, clients connected to channels for which that server had responsibility automatically reconnect to another server in that group. The view of a channel conversation that a particular user has, remains the same even through several reconnects to different servers.

As early as 1998, the Slashdot staff had set up an IRC network, called Slashnet. Initially this included a work channel for the staff to communicate with each other. This made sense since the staff was not always together in one place, but it was also just a natural form of communication for those with a Linux/open source background. A public channel was also added, for members of the Slashdot audience to communicate with the staff. The work channel quickly split into two channels, one for actual work communication, and another "water cooler" channel for idle conversation among staff members. Over time, other channels appeared, many from users treating Slashnet as just another IRC network, who were unaware that Slashnet and Slashdot were in any way affiliated.

By September of 2001, Slashnet had become an indispensable form of communication for the Slashdot staff. By this time, the staff was very distributed: Rob Malda and a core group of programmers remained in Holland, Michigan, but editors Timothy Lord and Robin Miller worked remotely; Timothy from various midwest locations, and Robin from Maryland. Jeff Bates had moved to Boston, working out of the offices of the parent company that had acquired Slashdot.[5]

Slashnet was, in many ways, the last refuge for Slashdot's original core audience. As the web site itself had become more mainstream, more about culture and less about technology, Slashnet represented a technical hard core of the site's open source roots. The barrier was not a very rigid one. While IRC channels can be moderated, and access can be password restricted, Slashnet, like most networks, was wide open for anyone to participate. In fact, though, the more mainstream online audience tended to gravitate to one-to-one IM systems like AIM or Yahoo! Instant Messenger, rather than the more text-based, more complex, and less user-friendly IRC.

Jeff Bates began the morning of September 11 at home in Boston before heading to the company office. He started with a call to Northwest Airlines, hoping to rearrange some business travel scheduled for later in the month. The call to Northwest was the first he knew that anything out of the ordinary was transpiring that day.

The woman at customer service told Jeff that a plane had flown into the World Trade Center. She had not seen or heard a news report directly, but was instead repeating what she had heard from other customers calling in that morning. Still on the phone, Jeff turned on the television to watch events unfolding on CNN, all the while describing what he was seeing to the woman at Northwest.

After talking to Northwest, Jeff called a friend's cell phone in Manhattan to make sure he was OK. This call went through; many others, from many other people that day, would not. Jeff left for the office wondering, as many people did in those early hours, if this was some sort of freak accident or something more sinister.

Nine hundred miles away, in Holland, Michigan, Rob Malda was also beginning his workday. For Rob, this involved logging on to the Slashnet staff IRC channel, checking his email, and reviewing the Slashdot submissions bin. Rob's first word of the World Trade Center attacks came from monitoring discussions on IRC. With no radio or television at hand, he attempted to look at the CNN and MSNBC web sites, but both sites were already struggling under heavy load, and other than one small, grainy photo from the CNN web site, Rob was unable to get any information. Only the first plane had hit at this point, but the Slashdot submissions bin was already filling with related submissions, and Rob quickly realized this was not going to be an ordinary news day.

Slashdot reviews submissions 24 hours a day, 7 days a week. To provide this coverage, the staff rotates who is in charge of the submissions bin. While any staff online at a given time can review submissions and make suggestions, one person has to be the final authority: only one person at a time can wear the pants in this family. That's an official Slashdot job description: "Daddy Pants." On the morning of September 11, Rob was wearing Daddy Pants. He made the decision that they would depart from their normal coverage and focus exclusively on the World Trade Center story.

By the time Jeff arrived in the Boston office, he had heard on the radio that the second plane had hit, and everyone knew that some form of terrorist attack was underway. Rob's decision to focus the coverage was the right one. By 9:30 A.M.. EST, Slashdot was serving 30-40 pages per second off of its six mirrored web servers, roughly double the usual traffic load.

It's significant to note the range of communication media the Slashdot staff was involved with during the first 90 minutes of that morning: land line telephone, cellphone, television, radio, web sites, email, and IRC. In his 1991 book Virtual Reality Howard Reingold described cyberspace as where you are when you're on the phone. His point, in part, was that we live in a vast and evermore pervasive telecommunications network, and that oftentimes our location within that network is more significant than our geographical location.

All of the Slashdot staff described that working day as one of feeling intimately connected to others at work, despite the fact that they were operating from at least five different geographical locations. In fact, neither Rob nor Jeff can recall, and probably never knew, where Timothy was that day. It could have been Texas, but it could just as easily have been Tennessee. All that mattered was that he was there in channel on IRC to contribute and help out.

It's also significant to note what part of the telecommunications network suffered that day. Cellphone calls in and out of New York and Washington became increasingly difficult, though some calls went through under remarkable and tragic circumstances. Cellphone calls provide our most intimate historical record of what happened within the World Trade Center itself, as well as what happened on the doomed Flight 93 that crashed in Pennsylvania. Land-line long-distance calls would experience bottlenecks nationally throughout the day. Television and radio provided important early reports, but these became less effective later in the day as reporters had difficulty getting on the scene.

News web sites suffered the most, many completely unprepared for the deluge of traffic. Slashdot began a daylong battle to stay up and stay on top of events. The Columbine experience had alerted them to the need to overhaul the site infrastructure. Many changes had been made; now those changes would be put to the test.

Thirty page views per second was well above Slashdot's normal load, but also about the limit of what the site architecture was designed to handle. Around 10:00 that morning, the backend database crashed, and the site was temporarily down. In fact, Slashdot had a backup database server on hand, one with a more current version of the database software (MySQL). This server was not yet online only because the staff didn't want to take the site offline to make the switch. The database crash provided an opportunity to quickly make that switch.

The new database server performed well, and now the bottleneck shifted to the web servers themselves. The staff made some on-the-fly adjustments to caching limits on the servers, and for the moment everything was functioning. It was now noon EST, and the site was serving 50 pages per second. Rob took a short break, and for the first time saw the actual video footage of the two crashes that morning.

As the staff struggled to keep the Slashdot web site up, a parallel phenomenon was emerging. Traffic on Slashnet was swelling, as more and more people turned to IRC as a way to communicate. The 200,000 pages per hour Slashdot's web site was now serving was impressive enough. Yet at the same time, Slashnet had thousands, and perhaps as many as 20,000 simultaneous users sharing information even more rapidly. The Slashdot staff set up a moderated channel to bring some organization to the process, but unmoderated public channels were springing up as well. There was one channel for communicating with the staff, and another for just general discussion.

Many of the links Slashdot posted that day, and many of the inline comments and quotes that appeared in stories, were pulled directly out of IRC on Slashnet. At a time when major news networks had difficulty getting reporters to the scene, Slashdot had eyewitness accounts coming in over IRC. At a time when major news web sites could not keep up with the traffic or rapidly changing information, Slashdot persevered. Some it was information, but some of it was a matter of dispelling misinformation ("I heard a truck bomb went off outside the State Department".... "No, my dad works at State, I just spoke to him, and nothing like that's going on"). Some of the concerns were global ("Who's behind the attacks?"). Some of the concerns were terribly personal ("I have a friend/loved one/family member who works in midtown Manhattan...").

Behind the scenes, the Slashdot staff frantically stripped down functionality on the site to keep the bare minimum of processes running and the maximum number of pages flowing. Dynamic content was turned off. Reverse DNS lookup was turned off. Eventually the ad server was turned off. Logfiles were turned off. After the initial database failure early in the day, however, the site stayed up. At the peak, Slashdot was serving 70 pages per second. For the day, it served more than 3 million page views.[6]

As a nation, we've never faced the kind of global telecommunications breakdown that the Internet was architected to handle as gracefully as possible. We have, however, seen episodes like September 11 that put a sudden and unexpected strain on the telecommunications infrastructure, and where the graceful degradation for which the Internet allows, matters a great deal. Taking advantage of that inherent robustness, however, requires a communication medium that follows the same architectural design and a group of communicators comfortable using that medium. Slashdot's successful coverage that day would not have been possible without IRC, a protocol as distributed and robust as the Internet itself, and without Slashnet, a community of users who knew how to make the most of that medium.

The word disintermediation has been much abused in the online world. The idea is simple enough: where traditional news media "mediates" between audience and events, the directness of the Internet should make this traditional mediation unnecessary. In practice, disintermediation happens far less often and far less effectively than one might think. Several elements need to be in place. First, there must be a genuine community of like-minded communicators looking to interact directly. Second, the medium through which they interact must be, to the degree possible, both responsive and transparent. Finally, those enabling the medium must have the humility to do nothing more than facilitate.

With Columbine, the Slashdot staff learned very quickly that they were not presenting a story, but were instead in the midst of a story that was happening all around them. The most useful thing they could do was get out of the way and let the story happen, let the Slashdot community connect to each other. September 11 proved again the value of facilitating rather than mediating. By and large, the audience that day did not notice or care that it was Slashdot they were using as their medium. Any channel of communication that was real time, up-to-date, and available would have sufficed. September 11 revealed which communications channels were up to that challenge; which could be effective but informative; which could disintermediate when disintermediation was needed most.

Conclusion

Image:Open Sources 2.0_I_9_tt36.png


In his influential essay, "The Cathedral & the Bazaar," published in the original Open Sources, Eric Raymond offered an arresting metaphor to contrast the top-down approach of traditional software development with the more grass-roots nature of open source software development. Cathedral-style development happens in isolation from users and with rigid authority from the top. Bazaar-style development is more community driven, without a clear line between developer and user, and follows a more evolutionary design process.

People often mistake Raymond's metaphor, however. Too often the assumption is made that the open source development community is a legion of programmers with hundreds or thousands of contributions coming in to each project, as if somehow by sheer numbers open source will triumph against its proprietary competitors. The mistake in this view is to look at the open source community as a flat, homogeneous organization.

Organizations seldom have so simple a structure. Online communities are not accidental organizations, thrown together by geography or family ties. Online communities are a subset of intentional communities, groups formed by those with common interests seeking out like-minded peers and exploiting the low communication cost of the Internet to make those connections. These communities indeed have a structure and a hierarchy. They are not so much a bazaar as they are a tribe.

Slashdot today serves more than 3.5 million page views a day. It is tempting to think of its viewers as an audience, not a community. It is easy to think of its viewers as a bazaar-style, unorganized mass. However, facilitating successful many-to-many communication requires a more sophisticated view. Fundamentally, Slashdot is a community, with the complex hierarchy all online communities have.

Rob Malda and Jeff Bates have commented that they see themselves not as staff versus audience, but as one group with a continuum of privileges. At the base of this hierarchy are the "anonymous cowards," who can read stories, submit stories, and comment on stories, though their comments start from a lower ranking. Registered users start from a higher position, with comments initially moderated higher, and with their activity tracked and evaluated within the system. Moderators are selected from among registered users, as are metamoderators. The paid staff have access to the actual submissions bin, as well as access to activity data about users. Finally, the staff member currently wearing Daddy Pants has ultimate authority over what submissions are actually posted to the front page. Parallel to all of this is the Slashnet IRC network. Some channels are moderated. Some are private. Some are public and open. Some moderated channels are moderated by Slashdot staff. Others are moderated and cover topics that have nothing to do with Slashdot itself. Some participants on IRC see Slashnet as an important way to connect to the larger Slashdot community. Others think of themselves as part of the Slashdot community, but never participate on IRC. Yet as September 11 revealed, even those uninvolved in or unaware of the IRC part of the community nonetheless benefit from it.

The system is authoritarian. So too is a tribe. Every position in this community has its unique privileges, however. Staff members do not have the right to moderate or metamoderate. Only registered users can do that. Furthermore, the system, though authoritarian, relies essentially on a practice of "term limits." A moderator receives a small number of moderation points, and once these are used, moderation rotates to someone else. Metamoderators get about 20 or so moderations to evaluate, and then metamoderation rotates to someone else. Even among the paid staff, no one has absolute editorial authority; Daddy Pants rotates among all of them.

Slashdot and other online communities, like many traditional tribes, bestow authority and privilege based on actions and reputations. This form of many-to-many communication works because these communities are egalitarian, but not democratic: everyone gets a voice, but not everyone gets a vote.

Notes

  1. If one thinks of the task of a moderator as similar to that of a copyeditor, it is possible to put an approximate monetary value on the work performed by users while moderating. If the typical moderator spends even an hour a day on moderation—and many spend much longer—this amounts to roughly $50,000 worth of work being done for the site for free each day. The key, though, is to see it in terms of value provided rather than money saved. Thanks to Slashdot's tiered moderation system, the site continues to scale and discussion continues to be valuable. There simple isn't anything one could spend $50,000 a day on to provide comparable value.
  2. The term troll does not originate with Slashdot, but in fact dates back to the early Usenet discussion forums on the Internet. Presumably it's a reference to the children's story "The Three Billy Goats Gruff," in which a troll lurks under the bridge waiting to ambush hapless passers by.
  3. The name Hellmouth derives from the television series "Buffy the Vampire Slayer." The series is set in the fictional town of Sunnydale, mainly at Sunnydale High, which sits atop a nexus of power drawing everything evil toward it; this nexus is known as the Hellmouth.
  4. Unique visitors (known in the trade as "uniques") is not a firm number. The industry norm is to count the number of IP numbers on client machines of visitors. This approach can undercount when visitors are behind a certain kind of proxies that shows only one IP number for everyone behind the proxy, and can overcount because not every IP number is associated with an actual person at the other end. The Slashdot staff uses a different method. Looking at historical data, Slashdot has an idea of the ratio of registered to unregistered visitors, as well as the page views per visit typical of registered and unregistered users. From this data, the number of unique visitors can be extrapolated. By industry norms, the number of daily "uniques" on Slashdot would be roughly 750,000.
  5. Originally this was online media company Andover.net. Andover was acquired by VA Linux Systems, and reformed as the wholly owned subsidiary OSDN, the Open Source Development Network. Since then, VA Linux Systems has changed its name to VA Software, and OSDN has changed its name to OSTG, the Open Source Technology Group.
  6. Rob Malda has nicely summarized the day's behind-the-scenes work in his piece, "Handling the Loads," at http://slashdot.org/article.pl?sid=01/09/13/154222.
Personal tools