Original Book was Copyright 2002 O'Reilly Media, Inc. All rights reserved.
Printed in the United States of America.
Published by O'Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O'Reilly Media books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safari.oreilly.com). For more information contact our corporate/institutional sales department: 800-998-9938 or .
Nutshell Handbook, the Nutshell Handbook logo, and the O'Reilly logo are registered trademarks of O'Reilly & Associates, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O'Reilly & Associates, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. The association between the image of a bare-throated bellbird and the topic of Jabber is a trademark of O'Reilly & Associates, Inc.
LEGO� is a registered trademark of The LEGO� Group. MINDSTORMS and Robotics Invention System are trademarks of The LEGO� Group. All other trademarks, service marks, and the like are the property of their owners.
While every precaution has been taken in the preparation of this book, the publisher and the author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.
What is Jabber? Depending on who you ask, the answer to that question could be any one (or all) of the following:
- Jabber is a technology.
- Jabber is a protocol (or set of protocols).
- Jabber is an XML-based Instant Messaging (IM) system.
- Jabber is an implementation of the set of protocols.
- Jabber is an idea whose time has come.
In fact, all these answers are right. Jabber is a set of protocols expressed within XML that allow people and applications to converse with one another. Sure, TCP sockets, STDIN/STDOUT, infrared, voice input, and teletype mechanisms all allow people and applications to converse; the difference is that Jabber provides a structured, extensible framework for exchanging all kinds of information.
This is all rather abstract, so what do we mean by "extensible framework for exchanging information"? Taking it one word at a time:
- Jabber's substrate is XML. XML is inherently extensible in the sense
- that tags can be added in a hierarchical sequence. Namespaces in XML
- allow us to keep track of the meaning and organization of these tags.
- You can put together a system for exchanging information using many
- different tools. The point about the "framework" is that the
- information exchanged and the entities that are exchanging it are
- contextualized, bringing meaning and structure to the interactions.
- Conversation is two-way, and it takes many forms
- question and answer, notification, compartmentalized discussion,
- and simple chat. Jabber supports all these different types of conversation and more.
- One doesn't really say that information is exchanged in a
- conversation, but when you bring applications into the mix of
- conversing entities, it may well be the case. And it's not just
- information exchanged in the form of conversations but also
- information about the entities themselves that flows across this
- context framework. It goes without saying: Jabber is an instant
- messaging (IM) system.
Instant messaging was Jabber's original raison d'�tre. Many deployments of Jabber software are to provide IM services, but Jabber is more than IM�certainly more than the phrase "instant messaging" represents. In this book you'll find out why this is so and how you can deploy solutions with Jabber that are more than mere chat. But most importantly, Jabber is fun!
Like chess, which has a small set of rules but countless game possibilities, the technologies employed in Jabber and the Jabber protocol itself are straighforward. The possibilities are almost limitless. Furthermore, because of a fundamental design feature (you might call it a "philosophical" angle), implementing Jabber-based solutions to your problems can be fun�really!
The History of Jabber
Jeremie Miller started the Jabber project early in 1998, and it was announced to the public in January 1999. To understand why Jabber came about, and in the form it took, let's look briefly at what existed in the IM world before Jabber.
The Pre-Jabber History
While the concept of live chat systems has existed for many years, with commands such as Unix's talk and VMS's phone, IM as we know it today existed as a concept and a handful of systems from companies such as Mirabilis, AOL, Microsoft, and Yahoo!. These systems (ICQ from Mirabilis, AIM from AOL, MSN from Microsoft, and Yahoo!IM from Yahoo!) allowed their users to chat to one another and avail themselves of IM-related services. However, an AIM user couldn't chat with an ICQ user, and MSN users couldn't interact with Yahoo!IM users. Each system was effectively closed to the outside world.
Furthermore, the protocols that these systems used were also mostly closed�proprietary�which meant it was difficult to find clients for these IM systems other than the ones supplied by the IM system owner.
Finally, the systems themselves were monolithic: multiple clients but a single server (or server farm). Although the companies were able to invest time and money into the problem, the fact remained that a monolithic architecture presented a scaling problem. Perhaps more relevant than that, companies that wanted to use IM services internally had to accept the fact that the conversations would be carried through the systems of a third party�namely the owners of these public IM systems. This was no more desirable than for a company to run its internal email using a public email service such as Hotmail.
Of course, these systems did have their advantages. The clients were accomplished and easy to learn and use, and as long as your correspondents were using the same IM system and you didn't mind your messages being carried by another organization (for private individuals these wouldn't be unique circumstances; again, we are led to the email services parallel), then you could leave the system management to someone else and get on with chatting.
Scratching an Itch
Having all your contacts use the same IM system is all well and good in theory but in practice is rarely the case. (If you're like me and have only a few friends, then this is not so much of a problem.) Jeremie Miller had correspondents in different IM systems and consequently had to have different IM clients running on his desktop to keep up with them all. Many great software projects stem from a personal "itch" that someone wanted to scratch. This was the primary itch that Jeremie had�a single client for all IM interaction: panacea.
Of course, one obvious solution would be to build a single client that supported all of the IM system protocols, but this approach had two drawbacks:
- The proprietary nature of the protocols made it harder to implement the support required and would make the client overly complicated.
- Every time the protocol, which wasn't under his control, changed or a new one came along, the client would have to be modified�a task not practical for a large user base. On top of that, GUI programming isn't everyone's cup of tea, and Jeremie preferred a solution that allowed him to concentrate on the underlying problems at hand and let others build the GUIs.
Along Came Jabber
So Jeremie resolved to create a solution that had the following characteristics:
It would have its own internal protocol, based upon XML. This protocol should be:
- Simple to understand and implement
- Easy to extend
- The complexity of bridging the disparate proprietary IM protocols would remain at the server, each bridge being a plug-in module.
- All the clients would have to implement only the single, simple open protocol; everything else would be implemented at the server. He called this solution "Jabber."
The main architectural feature for Jabber that Jeremie strived for was that it should be simple enough for anyone to implement a Jabber server of his own. Unlike a centralized server environment, with all of the traffic routed through a central point, Jeremie envisioned that Jabber would be decentralized, allowing individuals, companies, and public organizations to run their own servers. This is particularly relevant for internal-only, IM-style corporate communications. Just as email servers are used to exchange mail using the Simple Mail Transport Protocol (SMTP), the Jabber servers are able to connect and exchange IM traffic whenever necessary. Figure P-1 illustrates Jabber's distributed architecture, with two separate Jabber servers serving separate users.
Being open meant that Jabber could benefit from the help of anyone who wished to lend a hand, and administrators were empowered to be able to find and fix problems themselves if they so wished.
ß Being XML-based, as opposed to another binary format, for example, meant that the protocol streams were easy for humans to read, extensible, and readily integrated (a great range of XML parsing and construction tools is already available).
Being distributed meant that the Jabber system would belong to the people and that some of the scalability problems would be avoided. There remain some scalability issues, of course. Client-server communication that is TCP socket-based suffers from limitations of this technology. There are, however, initiatives to overcome these limitations with multiplexing techniques such as jpolld and dpsm (see http://download.jabber.org).
All of these features made for a good IM system design. But why stop at IM? Consider the client as an implementation of a simple protocol to exchange messages and presence information in XML structures and use plug-in services at the server, and what do you have? A language- and platform-agnostic XML routing framework.
Good grief, what a mouthful! This is why my response to "What is Jabber?" is usually just:
A really great technology!
IM System Features
This book assumes you have a basic knowledge of features commonly found in IM systems. In case you don't, here's a brief rundown of features relevant to what we'll be covering:
- In many cases, there's not much point in sending a quick message to
- someone if they're not there. Presence is a term used in IM to
- describe the technique of exchanging information, in a controlled
- manner, about availability (or unavailability).
- The idea is that
- when you connect to your IM server, your client sends an "I'm here"
- message that is relayed to your correspondents. It does the opposite
- when you disconnect. During the time you're connected, you can vary
- the information about your availability to reflect your immediate
- situation ("just popped out for coffee," "working on my resum��don't
- disturb me!").
- Buddy List�/roster
- Both terms (the former comes from the original IM systems, the latter
- from Jabber) refer to a list of correspondents with whom you regularly
- communicate and from whom you receive presence information. Depending
- on the IM system, the list may be stored on the client or on the
- server. Storing the list on the client has the (tenuous) advantage of
- being accessible when you're not connected to the server. Storing the
- list on the server means that you have a consistent roster content
- regardless of the client or workstation you happen to use�the list
- travels with you.
- Jabber stores the roster on the server.
- Push and pull
- When you connect to an IM system, there may be information the client
- needs to retrieve�pull�from the server (the roster, for instance).
- This is under direct control of your client as it decides when to make
- the retrieval. During the course of the connection, you'll receive
- messages from your correspondents. You don't request these messages by
- making a retrieve call to the server, as you would with the Post
- Office Protocol (POP) or Internet Message Access Protocol (IMAP) to
- retrieve email messages from the mail server; they're pushed to
- you as they occur. : In other words, you could say that the client
- must implement an event-based system, to listen out for and
- subsequently handle the incoming information, by displaying a pop-up
- window containing the chat message, for example. : The push/pull
- system lends itself well to traffic other than IM traffic.
- It almost goes without saying that, like other IM systems, Jabber has
- a client-server architecture. Clients and client libraries that
- implement the Jabber protocol, such as Net::Jabber,
- Jabber::Connection, Jabberpy, and
- JabberBeans, are available for many languages (here, for : Perl, Python, and Java).
- With Jabber, it is especially relevant to
- stress that the "weight balance" in complexity terms, between the
- client and the server, comes down heavily on the server side. Not only
- does this mean that the complexity remains where it should be�on the
- server�but also makes the task of writing clients easier and the
- resulting software lighter. ; Multiple versus single server
- We've already mentioned that the Jabber architecture does not dictate
- a single, centralized server. Not only does this mean that
- organizations can implement their own private system, but also that
- developers are free to install their own server and develop new
- plug-in services in addition to the IM bridges already available.
This book is not particularly about IM per se. Nor is it about the bridges to other IM systems. It's about the essence, the ideas, and the potential behind that concept and reality called Jabber.
You will learn about the Jabber protocol and how to use Jabber's technology to implement not only IM-based solutions but also solutions that don't involve inane chat. You'll learn how to install and configure your own Jabber server. You will discover more about the features of Jabber that give it its propensity for being an ideal messaging glue for many communication solutions; all of Jabber's technology features�the building blocks and the protocol itself�are explained; and you'll get to know how Jabber can be implemented in a variety of situations�some involving IM, others not�through a series of application and problem scenarios with fully working code examples, or recipes, in Perl, Python, and Java.
Here's a brief overview of what's in the book:
- Part I
- The first part of this book provides you with an introduction to
- Jabber; you'll learn about its features, why it's more than an IM
- system, and how to install and configure a Jabber server of your own.
- Chapter 1
- We begin with an imaginary conversation with human and application
- participants that shows how Jabber provides the supporting messaging
- "plasma." A short script shows how simple it is to make use of
- Jabber's power.
- Chapter 2
- We take a look at some of the features�the nature�of Jabber, to
- understand why Jabber is more than just an IM system. The features
- introduced in this chapter will be revisited as core building material
- for our recipes in Part I.
- Chapter 3
- Here you'll learn how to retrieve and install the Jabber server and
- perform minimal configuration, enough to be able to fire it up and use
- it as a basis for the recipes in Part I. Some troubleshooting and
- monitoring tips are also included.
- Chapter 4
- Once we have our Jabber server installed and running, we take a closer
- look at how the server has been designed. We focus on the server
- makeup and the different ways it can be deployed. A detailed tour of
- the standard configuration is also in this chapter.
- Part II
- The second part of this book provides detailed information about the
- Jabber protocol and a series of recipes�practical solutions to
- everyday problems�deployed in Jabber. The recipes use various Jabber
- features as a way of illustrating parts of the protocol.
- Chapter 5
- We take a detailed look at what Jabber looks like under the hood. We
- examine Jabber IDs (JIDs), resource and priority, and XML streams, and
- the basic Jabber building blocks (<message/>,
- <iq/>, and <presence/>).
- Chapter 6
- This chapter focuses on the standard Jabber namespaces used to
- accomplish IM-related and other tasks. A description for each one is
- given in turn.
- Chapter 7
- This chapter looks at the steps needed to create and authenticate with
- a user and examines the different types of authentication. We also
- build a utility to create users in order to generate users for our
- Chapter 8
- This chapter looks at some simple examples of Jabber deployment using
- basic features of message and presence, including presence
- Chapter 9
- Extending the themes introduced in Chapter 8, we look at how messages
- can be extended to carry custom and compartmentalized data and how to
- write a 'bot that serves in a conference room. Furthermore, we look at
- how components are defined and written and examine how different event
- models can coexist with Jabber's.
- Chapter 10
- This chapter shows diverse applications of Jabber, from building an
- online address book using Jabber as infrastructure, through exchanging
- XML-RPC-encoded requests and responses over Jabber, to extending the
- client scope of SAP's R/3 business software.
Software Used in This Book
The recipes in this book come in varying flavors, some in Perl, some in Python, and some in Java. These examples�to a greater or lesser degree�make use of prewritten libraries that provide at least the basic services needed to connect to a Jabber server and exchange data with it. Here's a summary of the versions of the languages used in this book, along with those libraries that are used, what features they offer, and where they're available. In addition, references to all of these libraries can be found on the Jabber development web site, http://dev.jabber.org. The installation instructions for the libraries can be found in the library packages themselves.
The Java recipes in this book are written in Java 2 (J2SE�the Java 2 Standard Edition), specifically with the 1.3.1 version of the Java Development Kit (JDK).
JabberBeans is the name of the Java library for Jabber used in this book. It offers comprehensive coverage of the features needed to write programs that interact with Jabber servers: connection, authentication, and the management of Jabber elements passed between your script and the Jabber server.
The JabberBeans library can be obtained from http://jabberbeans.org. The version used in this book is 0.9.0-pre4.
The recipes have been built and tested with Perl 5.6.0, although earlier and later versions of release 5 will probably work just fine.
Two libraries are available for programming Jabber solutions in Perl. Both come in the form of installable modules and are of the object-oriented persuasion.
- This module is available on the Comprehensive Perl Archive Network
- (CPAN), at http://www.cpan.org. Net::Jabber provides basic
- functionality for connecting to and interacting with a Jabber server,
- in addition to a host of higher-level features for manipulating all of
- the Jabber elements and making use of standard and custom namespaces.
- It relies upon a companion module XML::Stream, also
- available on CPAN, that provides the underlying mechanisms for
- creating connections to a Jabber server, as well as sending,
- receiving, and interpreting (parsing) the fragments of conversation
- between your script and that Jabber server.
- The version of
- Net::Jabber used in this book is 1.0022. The corresponding
- version of XML::Stream used is 1.12.
- The Jabber::Connection module is available on CPAN and
- provides the same basic features Net::Jabber does, albeit in
- a more "RISC" (Reduced Instruction Set Computing) way. While it
- provides similar functionality for connecting to and exchanging data
- with a Jabber server, it offers, via a companion modulecalled
- Jabber::NodeFactory, a lower-level API�similar to that in the
- Jabber server itself�for constructing and manipulating the Jabber
- elements. There are no high-level features; instead, you build your
- own using the building blocks that the module provides. : The version
- of Jabber::Connection used in this book is 0.02.
The Python examples have been written with Python 2.0.
Jabberpy is the name of the Python Jabber library used in the Python recipes in this book. As with Perl's Net::Jabber library set, Jabberpy provides its feature set from two separate libraries�jabber, which provides connectivity, authorization, and callback functions such as Net::Jabber and Jabber::Connection, and xmlstream, which provides the basic connectivity and parsing functions such as Net::Jabber's companion XML::Stream.
The Jabberpy libraries are available from its project site, at http://sourceforge.net/projects/jabberpy.
The version of Jabberpy used in this book is 0.2.
Where to Go for More Information
There are plenty of places out there on the Net to find out more about Jabber in general and programming with Jabber in particular. The main web site at jabber.org has a number of faces:
- This is the main entry point for all things Jabber. Start here if you
- want to find out about Jabber generally. ; http://dev.jabber.org
- You can find information on development efforts, both core and
- peripheral, here. There are lists of projects, discussion forums, and
- news items concerning new developments and initiatives in the Jabber
- world. ; http://docs.jabber.org
- This is the place to go to if you want more information on Jabber's
- technology and protocols. It has a range of documents contributed by
- various members of the Jabber community. In addition to the sites
- listed here, there are other resources available online for learning
- more about Jabber. Countless mailing lists, linked to from the
- http://dev.jabber.org page, cover subjects ranging from identity and
- profiles to security. The two main mailing lists are jadmin and
- jdev, covering administrativeand development subjects,
- respectively. All newcomers are welcome!
Finally, there's a Jabber conference room called jdev where many of the Jabber developers hang out. Just point your Jabber client at conference.jabber.org and drop by for a chat.
Conventions Used in This Book
The following typographical conventions are used in this book:
- Used for commands, programs, and options. All terms shown in bold are
- typed literally. ; Italic
- Used to show arguments and variables that should be replaced with
- user-supplied values. Italic is also used to indicate new terms, URLs,
- filenames, file extensions, directories, and to highlight comments in
- Constant Width
- Used to show the contents of files or the output from commands.
- Constant Width Bold
- Used in examples and tables to show commands or other text that should
- be typed literally by the user.
- Constant Width Italic
- Used in examples and tables to show text that should be replaced with
- user-supplied values.
How to Contact Us
We have tested and verified the information in this book to the best of our ability, but you may find that features have changed (or even that we have made mistakes!). Please let us know about any errors you find, as well as your suggestions for future editions, by writing:
- O'Reilly & Associates, Inc. 1005 Gravenstein Highway North : Sebastopol, CA 95472 (800) 998-9938 (in the U.S. or Canada) (707) : 829-0515 (international/local) (707) 829-0104 (fax) You can also send us messages electronically. To be put on the mailing list or to request a catalog, send email to:
-  To ask technical questions or comment on the book, send email to:
-  We have a web site for the book, where we'll list examples, errata, and any plans for future editions. The site also includes a link to a forum where you can discuss the book with the author and other readers. You can access this site at:
- http://www.oreilly.com/catalog/jabber For more information about this book and others, see the O'Reilly web site:
To my dearest wife, Sabine, who patiently and quietly took on all the demands of daily life for us during this project, allowing me to devote myself to researching and writing. To my son, Joseph, who knew I was writing a book but also knew that helping him build train tracks, drawing, and going on bicycle rides was always good tonic for a mind congested with XML.
To my grandpa, Harold Lomax, who never tired of reading through what I had written to check and correct grammar and punctuation.
To my editor, Chuck Toporek, who not only has great taste in music, but also has a sense of humor to match his infinite patience. Chuck taught me a great deal about writing and thinking, without me even realizing it was happening. Also, many thanks to David Chu, Chuck's editorial assistant. It's easy to miss the people behind the scenes when they do such a good job, so thanks for pushing my SGML files through when they needed to be checked by Tools, for coordinating the technical review, and for assisting Chuck to make this book happen. Thanks also to Lenny Muellner, my SGML guru. He helped me with my SGML queries and always kept me on the right track.
To the reviewers of this book, Jens Alfke, Matthew Allum, Michael Bauer, Piers Harding, Jeremie Miller, and Thomas Muldowney, who gave up a lot of their time to provide me with wonderful feedback and great insights. Special thanks go to Piers, the best technical debating partner you could wish for; trying to keep up with him mentally is what drives me on.
And, of course, where would this book be without the very thing that captured my imagination and held it all this time? Thanks to Jeremie and the team of core Jabber developers for building something so fascinating. Thanks also to the Jabber developer community at large, a more friendly bunch of diverse people one could hardly hope to meet.
Finally, thanks to Tim O'Reilly and all of O'Reilly & Associates, Inc., for seeing Jabber for what it really is and for having confidence in me to write about it.