Ubuntu Hacks/Administration

From WikiContent

< Ubuntu Hacks(Difference between revisions)
Jump to: navigation, search
(Initial conversion from Docbook)
Current revision (13:49, 7 March 2008) (edit) (undo)
(Initial conversion from Docbook)
 
(One intermediate revision not shown.)
Line 12: Line 12:
'''When you need to edit a configuration file from the command line in a pinch, use these tips for the ins and (especially) outs of vim.'''
'''When you need to edit a configuration file from the command line in a pinch, use these tips for the ins and (especially) outs of vim.'''
-
If one thing is for sure about Linux, it's that it has no shortage of text editors. This reflects the largely text-based nature of Linux, from the command line, to the source code, to the configuration files that programs refer to. When you are in a desktop environment, you can use one of many graphical tools to edit text files; however, in an emergency situation, or when logged into a machine remotely, you may need to fall back on a command-line text editor. Under Ubuntu, the default text editor is vim, and this hack provides you with the basic information you need to make changes to configuration files using vim.
+
If one thing is for sure about Linux, it's that it has no shortage of text editors. This reflects the largely text-based nature of Linux, from the command line, to the source code, to the configuration files that programs refer to. When you are in a desktop environment, you can use one of many graphical tools to edit text files; however, in an emergency situation, or when logged into a machine remotely, you may need to fall back on a command-line text editor. Under Ubuntu, the default text editor is ''vim'', and this hack provides you with the basic information you need to make changes to configuration files using ''vim''.
-
The vi editor has a rather mixed reputation. Some people love how quickly you can edit files with it, and others hate its steep learning curve. The vim editor (short for "Vi IMproved") takes vi and adds not only a number of powerful improvements, but also a number of user-friendly changes such as a complete, integrated help environment. Ubuntu includes vim by default, and even if you run vi, you are actually running vim in a vi-like emulation mode.
+
The ''vi'' editor has a rather mixed reputation. Some people love how quickly you can edit files with it, and others hate its steep learning curve. The ''vim'' editor (short for "Vi IMproved") takes ''vi'' and adds not only a number of powerful improvements, but also a number of user-friendly changes such as a complete, integrated help environment. Ubuntu includes ''vim'' by default, and even if you run ''vi'', you are actually running ''vim'' in a ''vi''-like emulation mode.
-
So, to edit a file, run vim on the command line followed by the filename as an argument. For instance, to edit the filesystem table (''/etc/fstab''), type:
+
So, to edit a file, run ''vim'' on the command line followed by the filename as an argument. For instance, to edit the filesystem table (''/etc/fstab''), type:
$ '''sudo vim /etc/fstab'''
$ '''sudo vim /etc/fstab'''
-
Unlike most other editors you may be used to, vim operates via a number of modes. Different keys perform different functions depending on which mode you are in. (This can be rather confusing the first time you use vim, since you will start typing only to see all sorts of strange and undesirable behavior.) For starters, there are two main modes you will be operating in: Normal mode (for navigating the file) and Insert mode (for editing text).
+
Unlike most other editors you may be used to, ''vim'' operates via a number of ''modes''. Different keys perform different functions depending on which mode you are in. (This can be rather confusing the first time you use ''vim'', since you will start typing only to see all sorts of strange and undesirable behavior.) For starters, there are two main modes you will be operating in: Normal mode (for navigating the file) and Insert mode (for editing text).
=== Navigate Through the File ===
=== Navigate Through the File ===
-
Normal mode is the main mode you default to when you start vim. In this mode, you use a number of different keys to move through the file. [[Ubuntu Hacks/Administration#ID-50863_TableLabel_Table8-1|Table 8-1]] lists some of the main navigation keys and their functions.
+
Normal mode is the main mode you default to when you start ''vim''. In this mode, you use a number of different keys to move through the file. [[Ubuntu Hacks/Administration#ID-50863_TableLabel_Table8-1|Table 8-1]] lists some of the main navigation keys and their functions.
<div id="ID-50863_TableLabel_Table8-1">
<div id="ID-50863_TableLabel_Table8-1">
-
'''Table 8-1. VIM Normal Mode keybindings'''
+
'''Table . VIM Normal Mode keybindings'''
{| border="1"
{| border="1"
Line 56: Line 56:
=== Copy and Paste ===
=== Copy and Paste ===
-
Sometimes when editing a file it is useful to copy and paste one line or a section of a line to another line. There are a number of ways to do so, some faster than others, but for new vim users, one of the easiest ways is through vim's Visual mode. To select a portion of text, type Esc until you are in Normal mode, move to the beginning of the text you want to copy, and then hit the v key to switch to Visual mode. Now move the cursor as normal, and you will notice that characters will be selected as it moves. When you have selected all of the characters you want, hit the y key to copy (or yank) the text. Then move to where you want to paste it and hit p.
+
Sometimes when editing a file it is useful to copy and paste one line or a section of a line to another line. There are a number of ways to do so, some faster than others, but for new ''vim'' users, one of the easiest ways is through ''vim'''s Visual mode. To select a portion of text, type Esc until you are in Normal mode, move to the beginning of the text you want to copy, and then hit the v key to switch to Visual mode. Now move the cursor as normal, and you will notice that characters will be selected as it moves. When you have selected all of the characters you want, hit the y key to copy (or yank) the text. Then move to where you want to paste it and hit p.
<div class="tip">
<div class="tip">
Line 66: Line 66:
=== Undo ===
=== Undo ===
-
Once nice feature that vim has that vi doesn't is unlimited undos. If you make a mistake and want to undo it, hit Esc until you are in Normal mode, and then simply type the u key to step back through each edit. If you find you need to redo a change, type Ctrl-R.
+
Once nice feature that ''vim'' has that ''vi'' doesn't is unlimited undos. If you make a mistake and want to undo it, hit Esc until you are in Normal mode, and then simply type the u key to step back through each edit. If you find you need to redo a change, type Ctrl-R.
=== Save and Exit ===
=== Save and Exit ===
-
After you have made your changes and are ready to save, you need to switch to Command mode. Command mode lets you run special vim commands, including the commands you need to save changes and exit. To save (or write) your changes, type <tt>:w</tt> and press Enter. When you are ready to quit, type <tt>:q</tt> and press Enter. You can also combine these two commands into a single command, and type <tt>:wq</tt> and press Enter.
+
After you have made your changes and are ready to save, you need to switch to Command mode. Command mode lets you run special ''vim'' commands, including the commands you need to save changes and exit. To save (or write) your changes, type <tt>:w</tt> and press Enter. When you are ready to quit, type <tt>:q</tt> and press Enter. You can also combine these two commands into a single command, and type <tt>:wq</tt> and press Enter.
=== Search and Replace ===
=== Search and Replace ===
-
A common need when editing files is to substitute one word for another throughout the file. To do so in vim, type:
+
A common need when editing files is to substitute one word for another throughout the file. To do so in ''vim'', type:
:s/''word''/''replacement''/g
:s/''word''/''replacement''/g
Line 86: Line 86:
:%s/''word''/''replacement''/gc
:%s/''word''/''replacement''/gc
-
and vim will ask you before it makes any changes.
+
and ''vim'' will ask you before it makes any changes.
One particularly handy use of the search-and-replace feature when editing configuration files is for commenting or uncommenting multiple lines in the file. To do so, enter Visual mode, select each line you want to comment, and then type:
One particularly handy use of the search-and-replace feature when editing configuration files is for commenting or uncommenting multiple lines in the file. To do so, enter Visual mode, select each line you want to comment, and then type:
Line 100: Line 100:
=== Help ===
=== Help ===
-
If you find you are lost, or you just want some help with vim's options, type:
+
If you find you are lost, or you just want some help with ''vim'''s options, type:
:help
:help
-
to enter vim's help screen. You can also follow <tt>:help</tt> with a particular keyword, so if you want to find out what the w key does, for instance, type:
+
to enter ''vim'''s help screen. You can also follow <tt>:help</tt> with a particular keyword, so if you want to find out what the w key does, for instance, type:
:help w
:help w
Line 110: Line 110:
=== Where to Learn More ===
=== Where to Learn More ===
-
This hack is by no means an exhaustive tutorial on how to use vim. vim has a number of advanced and powerful features that, once learned, allow you to make rapid changes to files. If you want to know more about vim's features and how to use them, type:
+
This hack is by no means an exhaustive tutorial on how to use ''vim''. ''vim'' has a number of advanced and powerful features that, once learned, allow you to make rapid changes to files. If you want to know more about ''vim'''s features and how to use them, type:
$ '''vimtutor'''
$ '''vimtutor'''
-
at the command line to try vim's built-in interactive tutorial program.
+
at the command line to try ''vim'''s built-in interactive tutorial program.
== Manage Users and Groups ==
== Manage Users and Groups ==
Line 156: Line 156:
''' username'''
''' username'''
-
''' '''
+
Line 167: Line 167:
</div>
</div>
-
To delete a user, use the deluser command:
+
To delete a user, use the ''deluser'' command:
$ '''sudo deluser'''
$ '''sudo deluser'''
''' username'''
''' username'''
-
''' '''
+
By default, this will remove only the user from the system, not any of the user's files. To remove the user's home directory and mail spool, add the <tt>--remove-home</tt> option. To remove all files on the system owned by the user no matter where they are, use the <tt>--remove-all-files</tt> option.
By default, this will remove only the user from the system, not any of the user's files. To remove the user's home directory and mail spool, add the <tt>--remove-home</tt> option. To remove all files on the system owned by the user no matter where they are, use the <tt>--remove-all-files</tt> option.
-
There are similar command-line commands to add and remove groups from the system. To add a group to the system, use the addgroup command:
+
There are similar command-line commands to add and remove groups from the system. To add a group to the system, use the ''addgroup'' command:
$ '''sudo addgroup'''
$ '''sudo addgroup'''
''' groupname'''
''' groupname'''
-
''' '''
+
Line 190: Line 190:
''' username'''
''' username'''
-
''' '''
+
''' groupname'''
''' groupname'''
-
''' '''
+
-
To remove a group from the system, use delgroup:
+
To remove a group from the system, use ''delgroup'':
$ '''sudo'''
$ '''sudo'''
Line 202: Line 202:
''' groupname'''
''' groupname'''
-
''' '''
+
Line 215: Line 215:
'''Tweak /etc/fstab to control which filesystems are mounted at boot.'''
'''Tweak /etc/fstab to control which filesystems are mounted at boot.'''
-
By default, Ubuntu will automatically detect and configure mount points for any partitions it finds when Ubuntu is installed. However, if you add a new disk to the system, or you want to automatically mount an NFS or SMB share at boot time, you must resort to the tried-and-true Linux method: editing ''/etc/fstab''.
+
By default, Ubuntu will automatically detect and configure ''mount points'' for any partitions it finds when Ubuntu is installed. However, if you add a new disk to the system, or you want to automatically mount an NFS or SMB share at boot time, you must resort to the tried-and-true Linux method: editing ''/etc/fstab''.
-
The ''/etc/fstab'' file (short for filesystem table) keeps track of filesystems that you want to mount in static locations. Here is a standard Ubuntu ''fstab'' file, which provides a good example for what each field in the file stands for:
+
The ''/etc/fstab'' file (short for ''filesystem table'') keeps track of filesystems that you want to mount in static locations. Here is a standard Ubuntu ''fstab'' file, which provides a good example for what each field in the file stands for:
# /etc/fstab: static file system information.
# /etc/fstab: static file system information.
Line 245: Line 245:
<div id="ID-27281_TableLabel_Table8-2">
<div id="ID-27281_TableLabel_Table8-2">
-
'''Table 8-2. Filesystems and their fstab labels'''
+
'''Table . Filesystems and their fstab labels'''
{| border="1"
{| border="1"
Line 278: Line 278:
-
In the fourth column, both fields list <tt>defaults</tt>, which mounts the filesystem with default settings. Different filesystems support different mount options, such as whether to mount the filesystem as read-only or read/write, the sorts of ownership that files in a partition might have, and so forth. We can get more information about special options that a filesystem supports from the mount manpage:
+
In the fourth column, both fields list <tt>defaults</tt>, which mounts the filesystem with default settings. Different filesystems support different mount options, such as whether to mount the filesystem as read-only or read/write, the sorts of ownership that files in a partition might have, and so forth. We can get more information about special options that a filesystem supports from the ''mount'' manpage:
$ '''man 8 mount'''
$ '''man 8 mount'''
-
In the fifth column, we left the values at <tt>0</tt> because we are not planning on using the dump utility to back up these partitions. This is a safe value for most partitions you might want to mount.
+
In the fifth column, we left the values at <tt>0</tt> because we are not planning on using the ''dump'' utility to back up these partitions. This is a safe value for most partitions you might want to mount.
-
In the final column, notice we set the Windows partition to <tt>0</tt> and the Linux partition to <tt>2</tt>. We did so because we don't want Linux to automatically run fsck on our Windows partition, yet we do want the Linux partition to be among the drives that are checked. We set the Linux partition's value to <tt>2</tt>, because the standard is to set the root filesystem (/) to <tt>1</tt> so that it is checked first, then set all other Linux filesystems to <tt>2</tt>.
+
In the final column, notice we set the Windows partition to <tt>0</tt> and the Linux partition to <tt>2</tt>. We did so because we don't want Linux to automatically run ''fsck'' on our Windows partition, yet we do want the Linux partition to be among the drives that are checked. We set the Linux partition's value to <tt>2</tt>, because the standard is to set the root filesystem (/) to <tt>1</tt> so that it is checked first, then set all other Linux filesystems to <tt>2</tt>.
=== Mount the Filesystems ===
=== Mount the Filesystems ===
-
Now that these two filesystems are configured, we can use the mount command to mount them:
+
Now that these two filesystems are configured, we can use the ''mount'' command to mount them:
$ '''sudo mount /mnt/windows1'''
$ '''sudo mount /mnt/windows1'''
Line 295: Line 295:
-
While we could have used the mount command to mount these filesystems manually, without bothering to add any entries to ''fstab'', when we set up the filesystems in ''fstab'', we don't have to list a lot of options—just the device or mount point we want to mount. After mounting our devices, we can type either <tt>mount</tt> or <tt>df</tt> to see all of our mounted filesystems.
+
While we could have used the ''mount'' command to mount these filesystems manually, without bothering to add any entries to ''fstab'', when we set up the filesystems in ''fstab'', we don't have to list a lot of options—just the device or mount point we want to mount. After mounting our devices, we can type either <tt>mount</tt> or <tt>df</tt> to see all of our mounted filesystems.
=== Mount Network Filesystems ===
=== Mount Network Filesystems ===
-
Network filesystems use slightly different syntax than ordinary partitions, so they get a separate section here. Specifically, the syntax you use to describe the filesystem (the first column) is different. For this example, we have an NFS share on host ''file1'' at ''/mnt/shares'', and a SMB file share on host file2 called ''data''. We want to mount the NFS share at ''/mnt/share1'' and the SMB share at ''/mnt/data1''. To mount both of these partitions at boot time, we would add the following lines to our ''/etc/fstab'':
+
Network filesystems use slightly different syntax than ordinary partitions, so they get a separate section here. Specifically, the syntax you use to describe the filesystem (the first column) is different. For this example, we have an NFS share on host ''file1'' at ''/mnt/shares'', and a SMB file share on host ''file2'' called ''data''. We want to mount the NFS share at ''/mnt/share1'' and the SMB share at ''/mnt/data1''. To mount both of these partitions at boot time, we would add the following lines to our ''/etc/fstab'':
file1:/mnt/shares /mnt/share1 nfs defaults 0 0
file1:/mnt/shares /mnt/share1 nfs defaults 0 0
Line 318: Line 318:
-
As with local partitions, we can use the mount or df commands to check the status of these mount points.
+
As with local partitions, we can use the ''mount'' or ''df'' commands to check the status of these mount points.
-
For more information on the syntax for the /etc/fstab file, and what sorts of mount options you can use for particular filesystems as well as what filesystems Linux can mount, see the ''fstab(5)'' and ''mount(8)'' manpages, respectively.
+
For more information on the syntax for the ''/etc/fstab'' file, and what sorts of mount options you can use for particular filesystems as well as what filesystems Linux can mount, see the ''fstab(5)'' and ''mount(8)'' manpages, respectively.
== Control Startup Services ==
== Control Startup Services ==
Line 345: Line 345:
=== Command-Line Method ===
=== Command-Line Method ===
-
Before learning how to start, stop, and disable services from the command line, it's important to understand Ubuntu's startup process and how Ubuntu determines which programs to run when it starts. For most Linux distributions (including Ubuntu), System V init scripts govern which programs start at boot and which programs don't. All System V init scripts that could potentially be run at boot are typically located in the ''/etc/init.d/'' directory for Ubuntu. Not every script in ''/etc/init.d'' is executed at boot, however. Linux organizes which scripts to run for different circumstances into runlevels; most Linux systems have seven runlevels, ranging from 0 to 6. Think of a runlevel as a checklist of programs for Ubuntu to start before it presents a login.
+
Before learning how to start, stop, and disable services from the command line, it's important to understand Ubuntu's startup process and how Ubuntu determines which programs to run when it starts. For most Linux distributions (including Ubuntu), ''System V init scripts'' govern which programs start at boot and which programs don't. All System V init scripts that could potentially be run at boot are typically located in the ''/etc/init.d/'' directory for Ubuntu. Not every script in ''/etc/init.d'' is executed at boot, however. Linux organizes which scripts to run for different circumstances into runlevels; most Linux systems have seven runlevels, ranging from 0 to 6. Think of a runlevel as a checklist of programs for Ubuntu to start before it presents a login.
A few of these runlevels are set aside for special states in a Linux system:
A few of these runlevels are set aside for special states in a Linux system:
Line 467: Line 467:
'''Special tools take the pain out of compiling and installing custom kernels.'''
'''Special tools take the pain out of compiling and installing custom kernels.'''
-
Debian boasts some very useful tools to make the job of building and installing kernels much easier, and of course those same tools are available in Ubuntu as well. They streamline the process of compiling a custom kernel and building a .deb package around it, allowing you to install a new kernel the same way you would any other package. This makes it possible to build a kernel on one machine and then just install the package on other machines, without having to recompile or track down all the loose ends—great if you need to upgrade a number of similar machines!
+
Debian boasts some very useful tools to make the job of building and installing kernels much easier, and of course those same tools are available in Ubuntu as well. They streamline the process of compiling a custom kernel and building a ''.deb'' package around it, allowing you to install a new kernel the same way you would any other package. This makes it possible to build a kernel on one machine and then just install the package on other machines, without having to recompile or track down all the loose ends—great if you need to upgrade a number of similar machines!
To build a custom kernel "the Ubuntu way," you will need to get hold of the kernel source and a variety of tools to configure, compile, and package it.
To build a custom kernel "the Ubuntu way," you will need to get hold of the kernel source and a variety of tools to configure, compile, and package it.
Line 517: Line 517:
: The most basic approach, this will just ask you a whole heap of questions, one after another. Personally, we hate configuring the kernel this way, because it's sequential and you have to go through a heap of stuff you probably couldn't care less about. Only bother with this as a last resort, such as if other options aren't available to you for some reason.
: The most basic approach, this will just ask you a whole heap of questions, one after another. Personally, we hate configuring the kernel this way, because it's sequential and you have to go through a heap of stuff you probably couldn't care less about. Only bother with this as a last resort, such as if other options aren't available to you for some reason.
;<tt>make menuconfig</tt>
;<tt>make menuconfig</tt>
-
: This is the most common way to do the configuration. It displays a nice, keyboard-driven menu that you can navigate using arrows, Enter, and the spacebar. However, one catch that can trip you up is that you need to have the development libraries for Ncurses installed, not just Ncurses itself. That's why we had you install the libncurses5-dev package a few paragraphs ago!
+
: This is the most common way to do the configuration. It displays a nice, keyboard-driven menu that you can navigate using arrows, Enter, and the spacebar. However, one catch that can trip you up is that you need to have the development libraries for Ncurses installed, not just Ncurses itself. That's why we had you install the ''libncurses5-dev'' package a few paragraphs ago!
<dl>
<dl>
Line 528: Line 528:
and all should be well.
and all should be well.
</dl>
</dl>
 +
=== Configuring the Kernel ===
=== Configuring the Kernel ===
Line 563: Line 564:
''' kernel-image-2.6.15_10.00.Custom_i386.deb'''
''' kernel-image-2.6.15_10.00.Custom_i386.deb'''
-
''' '''
+
Line 607: Line 608:
''' kernel-package'''
''' kernel-package'''
-
''' '''
+
Line 620: Line 621:
=== Get Simple Backup ===
=== Get Simple Backup ===
-
The first step is to install Simple Backup. Its package is called sbackup and is available from the ''universe'' repository, so if you haven't yet enabled ''universe'', you will need to do so. If you need more instructions on how to enable repositories, refer to [[Ubuntu Hacks/Package Management#Modify the List of Package Repositories|"Modify the List of Package Repositories" [Hack #60]]] or the individual hacks corresponding to your preferred package-management tool [Hacks [[Ubuntu Hacks/Package Management#Manage Packages from the Command Line|#54]], [[Ubuntu Hacks/Package Management#Manage Packages with Synaptic|#55]], and [[Ubuntu Hacks/Package Management#Manage Packages with Adept|#56]]].
+
The first step is to install Simple Backup. Its package is called ''sbackup'' and is available from the ''universe'' repository, so if you haven't yet enabled ''universe'', you will need to do so. If you need more instructions on how to enable repositories, refer to [[Ubuntu Hacks/Package Management#Modify the List of Package Repositories|"Modify the List of Package Repositories" [Hack #60]]] or the individual hacks corresponding to your preferred package-management tool [Hacks [[Ubuntu Hacks/Package Management#Manage Packages from the Command Line|#54]], [[Ubuntu Hacks/Package Management#Manage Packages with Synaptic|#55]], and [[Ubuntu Hacks/Package Management#Manage Packages with Adept|#56]]].
Once the program is installed, two new menu entries will appear under System→Administration: one called Simple Backup Config and a second called Simple Backup Restore. Select Simple Backup Config to see the fairly straightforward main window shown in [[Ubuntu Hacks/Administration#IDFigure8-5|Figure 8-5]].
Once the program is installed, two new menu entries will appear under System→Administration: one called Simple Backup Config and a second called Simple Backup Restore. Select Simple Backup Config to see the fairly straightforward main window shown in [[Ubuntu Hacks/Administration#IDFigure8-5|Figure 8-5]].
Line 729: Line 730:
=== Export the List of Installed Packages ===
=== Export the List of Installed Packages ===
-
The first step in cloning an installation is to grab the complete list of installed packages from the first system. To do so, you basically instruct dpkg to dump its entire list of packages, filter out any packages that aren't currently installed, and then redirect the output to a file:
+
The first step in cloning an installation is to grab the complete list of installed packages from the first system. To do so, you basically instruct ''dpkg'' to dump its entire list of packages, filter out any packages that aren't currently installed, and then redirect the output to a file:
$ '''<nowiki>sudo dpkg --get-selections | grep '[[:space:]]install$' | \\</nowiki>'''
$ '''<nowiki>sudo dpkg --get-selections | grep '[[:space:]]install$' | \\</nowiki>'''
Line 748: Line 749:
=== Import the Package List ===
=== Import the Package List ===
-
To import the package list, pipe the entire list to xargs, which then splits it into manageable chunks for the apt-get command:
+
To import the package list, pipe the entire list to ''xargs'', which then splits it into manageable chunks for the ''apt-get'' command:
$ '''cat package_list | xargs sudo apt-get install'''
$ '''cat package_list | xargs sudo apt-get install'''
Line 755: Line 756:
If you are migrating to a different Ubuntu release, this will require a bit of trial and error, since you will likely get complaints about packages no longer existing. The simple fix for this is to edit the package list and remove the package from the list, save your changes, and then rerun the command. You will likely need to do this a number of times, but eventually you will have a list of valid packages. A more complicated but thorough fix would involve checking the latest list of packages for a potential update or replacement for the package that no longer exists and installing them one by one.
If you are migrating to a different Ubuntu release, this will require a bit of trial and error, since you will likely get complaints about packages no longer existing. The simple fix for this is to edit the package list and remove the package from the list, save your changes, and then rerun the command. You will likely need to do this a number of times, but eventually you will have a list of valid packages. A more complicated but thorough fix would involve checking the latest list of packages for a potential update or replacement for the package that no longer exists and installing them one by one.
-
Once apt-get completes, you are finished. All of the files from the package list will have been imported into the new system. Now keep in mind that this doesn't mean that all of the settings have transferred over. To do that, you will likely need to copy settings from the ''/etc'' directory or possibly other directories, depending on the package.
+
Once ''apt-get'' completes, you are finished. All of the files from the package list will have been imported into the new system. Now keep in mind that this doesn't mean that all of the settings have transferred over. To do that, you will likely need to copy settings from the ''/etc'' directory or possibly other directories, depending on the package.
=== Hacking the Hack ===
=== Hacking the Hack ===
Line 826: Line 827:
-
and see if you get a list of files somewhat similar to what you see in [[Ubuntu Hacks/Administration#IDFigure8-13|Figure 8-13]] has sample df output you can reference. On a default Ubuntu install, your root filesystem will likely be on ''/dev/discs/disc0/part1''.
+
and see if you get a list of files somewhat similar to what you see in [[Ubuntu Hacks/Administration#IDFigure8-13|Figure 8-13]] has sample ''df'' output you can reference. On a default Ubuntu install, your root filesystem will likely be on ''/dev/discs/disc0/part1''.
</dl>
</dl>
 +
<div id="IDFigure8-13">
<div id="IDFigure8-13">
Line 838: Line 840:
<dl>
<dl>
<dt>Repair the filesystem
<dt>Repair the filesystem
-
<dd>Now use the fsck tool to attempt to repair your filesystem. First, you will need to unmount the ''/target'' partition if it is currently mounted:
+
<dd>Now use the ''fsck'' tool to attempt to repair your filesystem. First, you will need to unmount the ''/target'' partition if it is currently mounted:
~ # '''umount /target'''
~ # '''umount /target'''
-
The rescue-mode environment has fsck utilities only for ''ext2'', ''ext3'', and ''jfs'', so if you chose another filesystem during the install, you will have to resort to a different rescue disk. Ubuntu defaults to ''ext3'', so if you went with the default, type the following to check and repair your drive:
+
The rescue-mode environment has ''fsck'' utilities only for ''ext2'', ''ext3'', and ''jfs'', so if you chose another filesystem during the install, you will have to resort to a different rescue disk. Ubuntu defaults to ''ext3'', so if you went with the default, type the following to check and repair your drive:
~ # '''/sbin/'''
~ # '''/sbin/'''
''' fsck.ext3'''
''' fsck.ext3'''
-
''' '''
+
'''-f'''
'''-f'''
''' /dev/discs/disc0/part1'''
''' /dev/discs/disc0/part1'''
-
''' '''
+
-
Replace <tt>''fsck.ext3''</tt> with <tt>fsck.ext2</tt> or <tt>fsck.jfs</tt> if you have an ''ext2'' or ''jfs'' partition, respectively, and replace the partition with your particular root partition. Once the fsck completes, attempt to mount the partition under ''/target'' again and try to access it with ls:
+
Replace <tt>''fsck.ext3''</tt> with <tt>fsck.ext2</tt> or <tt>fsck.jfs</tt> if you have an ''ext2'' or ''jfs'' partition, respectively, and replace the partition with your particular root partition. Once the ''fsck'' completes, attempt to mount the partition under ''/target'' again and try to access it with ''ls'':
~ # '''mount '''
~ # '''mount '''
''' /dev/discs/disc0/part1'''
''' /dev/discs/disc0/part1'''
-
''' '''
+
''' /target'''
''' /target'''
~ # '''ls /target'''
~ # '''ls /target'''
Line 867: Line 869:
If you see a number of files and directories, congratulations, your drive is repaired! Type <tt>exit</tt> to leave the BusyBox shell and reboot, and see if you can boot into your system successsfully.
If you see a number of files and directories, congratulations, your drive is repaired! Type <tt>exit</tt> to leave the BusyBox shell and reboot, and see if you can boot into your system successsfully.
</dl>
</dl>
 +
==== The system boots and continues through part of the boot process, but hangs at a particular service ====
==== The system boots and continues through part of the boot process, but hangs at a particular service ====
Line 891: Line 894:
;syslog
;syslog
-
: ''syslog'' is the primary system log and contains log output from daemons and other programs running on the system, such as dhclient, cron, init, ''xscreensaver'', and some kernel logs. This log is the first place to look when trying to track down general system errors.
+
: ''syslog'' is the primary system log and contains log output from daemons and other programs running on the system, such as ''dhclient'', ''cron'', ''init'', ''xscreensaver'', and some kernel logs. This log is the first place to look when trying to track down general system errors.
;dmesg
;dmesg
: This log traditionally lists all of the boot-time kernel logging for a system, along with any other kernel logs related to devices and module loading. Check here to see what sorts of devices the kernel detected at boot time, as well as to track down any errors the kernel might have had when loading a module.
: This log traditionally lists all of the boot-time kernel logging for a system, along with any other kernel logs related to devices and module loading. Check here to see what sorts of devices the kernel detected at boot time, as well as to track down any errors the kernel might have had when loading a module.
Line 899: Line 902:
: The Xorg logs contain very detailed output from Xorg and are numbered according to which display it is running. The default Xorg session runs on display 0, so for the most part you would look through ''Xorg.0.log'' to trace down errors in Xorg.
: The Xorg logs contain very detailed output from Xorg and are numbered according to which display it is running. The default Xorg session runs on display 0, so for the most part you would look through ''Xorg.0.log'' to trace down errors in Xorg.
;messages
;messages
-
: This log contains some kernel log messages along with log output from certain system programs. For instance, gconfd and some other programs log here.
+
: This log contains some kernel log messages along with log output from certain system programs. For instance, ''gconfd'' and some other programs log here.
;daemon.log
;daemon.log
: Here, you will find the same daemon output that you would normally find in ''syslog'', but without log output from kernels or other systems that log to ''syslog''.
: Here, you will find the same daemon output that you would normally find in ''syslog'', but without log output from kernels or other systems that log to ''syslog''.
;auth.log
;auth.log
-
: Inside ''auth.log'', you will find information about user authentication including every time a user logs in to the system or executes sudo.
+
: Inside ''auth.log'', you will find information about user authentication including every time a user logs in to the system or executes ''sudo''.
;mail.log
;mail.log
: If you use your system as an email server, this log will contain information about incoming and outgoing messages, along with any errors.
: If you use your system as an email server, this log will contain information about incoming and outgoing messages, along with any errors.
Line 935: Line 938:
'''Warning'''
'''Warning'''
-
Some people might be tempted to just use vi when opening logfiles; however, be warned that vi caches the entire file to disk when it opens it. This might not be a problem for small logfiles, but when you open a 2 GB Apache log, you might possibly fill up the remaining space on your disk! less does not cache the entire file like vi, so it is the safer choice for large logfiles.
+
Some people might be tempted to just use ''vi'' when opening logfiles; however, be warned that ''vi'' caches the entire file to disk when it opens it. This might not be a problem for small logfiles, but when you open a 2 GB Apache log, you might possibly fill up the remaining space on your disk! ''less'' does not cache the entire file like ''vi'', so it is the safer choice for large logfiles.
</div>
</div>
-
grep is also a very useful tool for logs. grep accepts a pattern as an argument and will look for that pattern within a file. For instance:
+
''grep'' is also a very useful tool for logs. ''grep'' accepts a pattern as an argument and will look for that pattern within a file. For instance:
$ '''grep dhclient /var/log/syslog'''
$ '''grep dhclient /var/log/syslog'''
Line 953: Line 956:
'''Tip'''
'''Tip'''
-
Some logfiles you want to view might be ''gzip''ped due to log rotation. In this case, use zless and zgrep just as you would less and grep, respectively.
+
Some logfiles you want to view might be ''gzip''ped due to log rotation. In this case, use ''zless'' and ''zgrep'' just as you would ''less'' and ''grep'', respectively.
</div>
</div>
Line 1,119: Line 1,122:
''' sysfs_device_path'''
''' sysfs_device_path'''
-
''' '''
+
-
''' '''
+
''' subsystem'''
''' subsystem'''
-
''' '''
+
Line 1,177: Line 1,180:
'''If you can access it via SSH, you can mount it on your filesystem, giving you secure access to your favorite stuff.'''
'''If you can access it via SSH, you can mount it on your filesystem, giving you secure access to your favorite stuff.'''
-
Samba has transformed how Penguinistas interact on networks with Windows machines and even other Linux boxes. In particular, I often use smbfs (short for "Samba filesystem") to mount a Samba share on my box for easy access. For example, I have two computers on my network: a server named sophocles and a laptop named euripides. A huge collection of music can be found on ''sophocles'' at ''/var/music'', but I want easy access to those goodies from ''euripides''. Using Samba on ''sophocles'', I share ''/var/music'', and using smbfs on ''euripides'', I mount ''/var/music'' on my server ''sophocles'' to ''/home/scott/tunes'' on my laptop ''euripides''. By doing so, it appears to me while I'm using ''euripides'' as though ''tunes'' is just another local folder on my laptop, so I can read files from it, save to it, and do anything else I could if that folder were in fact on my machine.
+
Samba has transformed how Penguinistas interact on networks with Windows machines and even other Linux boxes. In particular, I often use ''smbfs'' (short for "Samba filesystem") to mount a Samba share on my box for easy access. For example, I have two computers on my network: a server named ''sophocles'' and a laptop named ''euripides''. A huge collection of music can be found on ''sophocles'' at ''/var/music'', but I want easy access to those goodies from ''euripides''. Using Samba on ''sophocles'', I share ''/var/music'', and using ''smbfs'' on ''euripides'', I mount ''/var/music'' on my server ''sophocles'' to ''/home/scott/tunes'' on my laptop ''euripides''. By doing so, it appears to me while I'm using ''euripides'' as though ''tunes'' is just another local folder on my laptop, so I can read files from it, save to it, and do anything else I could if that folder were in fact on my machine.
This is great, except that there are some issues. Setting up Samba can be a royal pain, so any time I can use something simpler, I jump at the chance. Second, Samba shares aren't secure by default. Call me paranoid, but I don't like anything flowing over a network that isn't encrypted. Yes, it's possible to tunnel Samba using SSH, but that just adds more time and trouble on top of the royal pain that Samba sometimes causes. Finally, Samba was designed for LANs, not the wild and woolly Internet, so accessing shares remotely is out of the question (yes, there are ways to do it, but it's just not a good idea on today's Net...and it causes yet more complication!).
This is great, except that there are some issues. Setting up Samba can be a royal pain, so any time I can use something simpler, I jump at the chance. Second, Samba shares aren't secure by default. Call me paranoid, but I don't like anything flowing over a network that isn't encrypted. Yes, it's possible to tunnel Samba using SSH, but that just adds more time and trouble on top of the royal pain that Samba sometimes causes. Finally, Samba was designed for LANs, not the wild and woolly Internet, so accessing shares remotely is out of the question (yes, there are ways to do it, but it's just not a good idea on today's Net...and it causes yet more complication!).
-
But I'm here to tell you that there's a better way: sshfs, the SSH filesystem. Instead of Samba, it uses SSH. All the problems I listed previously are obviated when you switch to SSH. SSH is a breeze to set up and use. All SSH traffic is encrypted, hence the name Secure SHell. And SSH was designed for use on LANs as well as the big, bad Internet, so it's a great choice no matter where you are.
+
But I'm here to tell you that there's a better way: ''sshfs'', the SSH filesystem. Instead of Samba, it uses SSH. All the problems I listed previously are obviated when you switch to SSH. SSH is a breeze to set up and use. All SSH traffic is encrypted, hence the name Secure SHell. And SSH was designed for use on LANs as well as the big, bad Internet, so it's a great choice no matter where you are.
-
To go back to my original example, I can SSH from ''euripides'' to ''sophocles'' (and vice versa, for that matter), so I now use sshfs to mount ''/var/music'' on ''sophocles'' to ''/home/scott/tunes'' on ''euripides''. It's easy, it's encrypted, and if this is a connection I'm going to need all the time, I can set things up in ''/etc/fstab'' to automate the whole process. Ready to try it out?
+
To go back to my original example, I can SSH from ''euripides'' to ''sophocles'' (and vice versa, for that matter), so I now use ''sshfs'' to mount ''/var/music'' on ''sophocles'' to ''/home/scott/tunes'' on ''euripides''. It's easy, it's encrypted, and if this is a connection I'm going to need all the time, I can set things up in ''/etc/fstab'' to automate the whole process. Ready to try it out?
Before doing anything else, make sure that you can SSH from the client to the server:
Before doing anything else, make sure that you can SSH from the client to the server:
Line 1,208: Line 1,211:
If you can successfully SSH in to the server, you're finished with that machine (see how much simpler this is than it would be if you were using Samba?).
If you can successfully SSH in to the server, you're finished with that machine (see how much simpler this is than it would be if you were using Samba?).
-
Now you need to focus on the local computer. Make sure you've enabled the ''universe'' archive in your ''/etc/apt/sources.list'' file, as detailed in [[Ubuntu Hacks/Package Management#Modify the List of Package Repositories|"Modify the List of Package Repositories" [Hack #60]]]. Once you've done so, install sshfs:
+
Now you need to focus on the local computer. Make sure you've enabled the ''universe'' archive in your ''/etc/apt/sources.list'' file, as detailed in [[Ubuntu Hacks/Package Management#Modify the List of Package Repositories|"Modify the List of Package Repositories" [Hack #60]]]. Once you've done so, install ''sshfs'':
$ '''sudo apt-get install sshfs'''
$ '''sudo apt-get install sshfs'''
-
You'll be told that apt is going to have to install some additional packages in addition to sshfs (fuse-utils and libfuse2), so go ahead and approve apt's request.
+
You'll be told that ''apt'' is going to have to install some additional packages in addition to ''sshfs'' (''fuse-utils'' and ''libfuse2''), so go ahead and approve ''apt'''s request.
-
When apt finishes installing your new software, it's time to fix some permissions so that normal users can mount and unmount using sshfs. If you leave out this step, mounting and unmounting will require the use of sudo and knowledge of the ''root'' password, which you probably don't want. Better to go ahead and run this command, which grants all users on your system execute permission for the fusermount command, used by sshfs to perform its magic:
+
When ''apt'' finishes installing your new software, it's time to fix some permissions so that normal users can mount and unmount using ''sshfs''. If you leave out this step, mounting and unmounting will require the use of ''sudo'' and knowledge of the ''root'' password, which you probably don't want. Better to go ahead and run this command, which grants all users on your system execute permission for the ''fusermount'' command, used by ''sshfs'' to perform its magic:
$ '''sudo chmod +x /usr/bin/fusermount'''
$ '''sudo chmod +x /usr/bin/fusermount'''
Line 1,225: Line 1,228:
-
In order for sshfs to work, you have to load the fuse module. For now, go ahead and run this command:
+
In order for ''sshfs'' to work, you have to load the ''fuse'' module. For now, go ahead and run this command:
$ '''sudo modprobe fuse'''
$ '''sudo modprobe fuse'''
-
I know you don't want to have to run that command every time you want to mount with sshfs, so use the following command to tell Ubuntu to automatically load the fuse module when you boot:
+
I know you don't want to have to run that command every time you want to mount with ''sshfs'', so use the following command to tell Ubuntu to automatically load the ''fuse'' module when you boot:
$ '''sudo sh -c "echo 'fuse' >> /etc/modules"'''
$ '''sudo sh -c "echo 'fuse' >> /etc/modules"'''
Line 1,246: Line 1,249:
''' user@'''
''' user@'''
-
''' '''
+
'''['''
'''['''
''' IP'''
''' IP'''
-
''' '''
+
'''|'''
'''|'''
''' HOSTNAME of remote machine'''
''' HOSTNAME of remote machine'''
-
''' '''
+
''']'''
''']'''
''' :/shared/path /local/mount/point'''
''' :/shared/path /local/mount/point'''
-
''' '''
+
-
If your username is the same on the remote and local machines, you can leave it off. DNS will work either on your LAN, if you have an entry for the server in your ''/etc/hosts'' file, or over the Net to a machine with a registered address. Otherwise, use the computer's IP address. If you don't specify a path after the colon, sshfs mounts the home directory for the SSH user, which may be all that you want. In my case, my username is the same on both boxes, I have an entry for ''sophocles'' in the ''/etc/hosts'' file, and I want to mount a specific directory on the computer, so my command looks like this:
+
If your username is the same on the remote and local machines, you can leave it off. DNS will work either on your LAN, if you have an entry for the server in your ''/etc/hosts'' file, or over the Net to a machine with a registered address. Otherwise, use the computer's IP address. If you don't specify a path after the colon, ''sshfs'' mounts the home directory for the SSH user, which may be all that you want. In my case, my username is the same on both boxes, I have an entry for ''sophocles'' in the ''/etc/hosts'' file, and I want to mount a specific directory on the computer, so my command looks like this:
$ '''sshfs sophocles:/var/music /home/scott/tunes'''
$ '''sshfs sophocles:/var/music /home/scott/tunes'''
Line 1,274: Line 1,277:
Now I can open my favorite music player, point it to ''/home/scott/tunes'', and start enjoying jazz, rock, country, or whatever it is that floats my boat. Remember, it's all secure, so I don't have to worry about anyone sniffing packets and finding out what's moving around on my network.
Now I can open my favorite music player, point it to ''/home/scott/tunes'', and start enjoying jazz, rock, country, or whatever it is that floats my boat. Remember, it's all secure, so I don't have to worry about anyone sniffing packets and finding out what's moving around on my network.
-
To unmount, you have a choice. You can use the fusermount command with the <tt>-u</tt> option (for "unmount"):
+
To unmount, you have a choice. You can use the ''fusermount'' command with the <tt>-u</tt> option (for "unmount"):
$ '''fusermount -u /home/scott/tunes'''
$ '''fusermount -u /home/scott/tunes'''
-
Or it might be simpler just to use the good ol' umount command:
+
Or it might be simpler just to use the good ol' ''umount'' command:
$ '''sudo umount /home/scott/tunes'''
$ '''sudo umount /home/scott/tunes'''
Line 1,288: Line 1,291:
sophocles:/var/music /home/scott/tunes fuse defaults 0 0
sophocles:/var/music /home/scott/tunes fuse defaults 0 0
-
The first three items are different from the usual ''fstab'' entries, with the first two exactly like the sshfs command you just used, and <tt>fuse</tt> indicating to your computer the kind of filesystem it's going to mount. If you want to change the other entries and you know what you're doing (run <tt>man</tt> <tt>fstab</tt> to learn more), go right ahead.
+
The first three items are different from the usual ''fstab'' entries, with the first two exactly like the ''sshfs'' command you just used, and <tt>fuse</tt> indicating to your computer the kind of filesystem it's going to mount. If you want to change the other entries and you know what you're doing (run <tt>man</tt> <tt>fstab</tt> to learn more), go right ahead.
-
Once you discover sshfs, you're going to use it all the time. There's no reason not to: it works beautifully, and the added benefit of security is icing on the cake. Thank you, sshfs!
+
Once you discover ''sshfs'', you're going to use it all the time. There's no reason not to: it works beautifully, and the added benefit of security is icing on the cake. Thank you, ''sshfs''!
Scott Granneman
Scott Granneman
Line 1,366: Line 1,369:
There's an old saying that everyone should take to heart: "There are two kinds of computer users: those who have lost data and those who are about to." I belong in that first camp, but it only took one nasty experience with an accidentally deleted term paper to force me to resolve never to let that happen again. That night long ago, when I lost the fruits of my labors analyzing the great film ''Raising Arizona'', taught me the hard way that backing up is essential. And while traditional backups [[Ubuntu Hacks/Administration#Back Up Your System|[Hack #79]]] are vital, keeping two or more machines in sync not only gives you some extra security, but it also makes it easy to switch from system to system without having to remember which one has the latest copy of a file.
There's an old saying that everyone should take to heart: "There are two kinds of computer users: those who have lost data and those who are about to." I belong in that first camp, but it only took one nasty experience with an accidentally deleted term paper to force me to resolve never to let that happen again. That night long ago, when I lost the fruits of my labors analyzing the great film ''Raising Arizona'', taught me the hard way that backing up is essential. And while traditional backups [[Ubuntu Hacks/Administration#Back Up Your System|[Hack #79]]] are vital, keeping two or more machines in sync not only gives you some extra security, but it also makes it easy to switch from system to system without having to remember which one has the latest copy of a file.
-
If a user new to Linux starts asking more experienced users about a good way to sync his data, he will soon hear about a wonderful tool named rsync. ''rsync'' was developed by Andrew "Tridge" Tridgell, the same man behind Samba; in fact, Tridge has stated that he believes he'll be remembered through posterity for ''rsync'' far more than for Samba, and he just may be right. However, ''rsync'', while absolutely excellent, is not necessarily the best software to use for syncing two repositories that you're actively working with.
+
If a user new to Linux starts asking more experienced users about a good way to sync his data, he will soon hear about a wonderful tool named ''rsync''. ''rsync'' was developed by Andrew "Tridge" Tridgell, the same man behind Samba; in fact, Tridge has stated that he believes he'll be remembered through posterity for ''rsync'' far more than for Samba, and he just may be right. However, ''rsync'', while absolutely excellent, is not necessarily the best software to use for syncing two repositories that you're actively working with.
=== The Problem with rsync ===
=== The Problem with rsync ===
Line 1,399: Line 1,402:
</div>
</div>
-
Click "Create new profile," and you're asked to give a name for the new profile. Since I'm going to sync up my folder that contains the work I've done on this book, I'm going to cleverly name this profile ubuntu_hacks and click OK. Now both profiles, default and ubuntu_hacks, appear. I double-click on ubuntu_hacks, and a small window opens, shown in [[Ubuntu Hacks/Administration#IDFigure8-18|Figure 8-18]], asking me to enter the local directory that I want to sync.
+
Click "Create new profile," and you're asked to give a name for the new profile. Since I'm going to sync up my folder that contains the work I've done on this book, I'm going to cleverly name this profile ''ubuntu_hacks'' and click OK. Now both profiles, ''default'' and ''ubuntu_hacks'', appear. I double-click on ''ubuntu_hacks'', and a small window opens, shown in [[Ubuntu Hacks/Administration#IDFigure8-18|Figure 8-18]], asking me to enter the local directory that I want to sync.
<div id="IDFigure8-18">
<div id="IDFigure8-18">
Line 1,419: Line 1,422:
</div>
</div>
-
Notice that I can sync to a local folder (which could also be a folder accessed through Samba or NFS that I've mounted on this machine), or to another machine via SSH, RSH, or a socket. In this case, I'm using SSH, so I enter the address of the remote host—in this case, 192.168.0.14—and the username, which is rsgranne. Finally, I type in the remote directory I want to sync:
+
Notice that I can sync to a local folder (which could also be a folder accessed through Samba or NFS that I've mounted on this machine), or to another machine via SSH, RSH, or a socket. In this case, I'm using SSH, so I enter the address of the remote host—in this case, 192.168.0.14—and the username, which is ''rsgranne''. Finally, I type in the remote directory I want to sync:
/media/data/zzz_rsg/documents/clientele/aaa_current/OReilly/ubuntu_hacks
/media/data/zzz_rsg/documents/clientele/aaa_current/OReilly/ubuntu_hacks
Line 1,435: Line 1,438:
=== Tweaking Your Profiles ===
=== Tweaking Your Profiles ===
-
When I first started Unison, I had to create a profile, which keeps track of the directories I want to compare and the transport method, among other configurations. Profiles live in a hidden directory (''.unison''), which itself is in your home directory, and they're just text files that end with the .prf extension. Before I finish looking at Unison, I want to mention a few other tweaks that you should add manually to those files in order to make your sync processes work a little more smoothly.
+
When I first started Unison, I had to create a profile, which keeps track of the directories I want to compare and the transport method, among other configurations. Profiles live in a hidden directory (''.unison''), which itself is in your home directory, and they're just text files that end with the ''.prf'' extension. Before I finish looking at Unison, I want to mention a few other tweaks that you should add manually to those files in order to make your sync processes work a little more smoothly.
In order to add these tweaks, you're going to need to open your profiles with your favorite text editor. In my case, I'm going to open and change ''ubuntu_hacks.prf'', located in ''/home/rsgranne/.unison''.
In order to add these tweaks, you're going to need to open your profiles with your favorite text editor. In my case, I'm going to open and change ''ubuntu_hacks.prf'', located in ''/home/rsgranne/.unison''.
Line 1,458: Line 1,461:
When you add these two settings to your profile, you order Unison to synchronize the owners and groups of files, along with their contents, which can really help to keep things straight as you move files around between machines.
When you add these two settings to your profile, you order Unison to synchronize the owners and groups of files, along with their contents, which can really help to keep things straight as you move files around between machines.
-
If you want to synchronize users and groups by the real numbers that your Ubuntu system uses instead of the names that we humans use (in other words, by 1000 instead of rsgranne; if you don't know what I'm talking about, take a look at your ''/etc/passwd'' file and run the command <tt>man</tt> <tt>5</tt> <tt>passwd</tt>), then add this preference to your profile as well:
+
If you want to synchronize users and groups by the real numbers that your Ubuntu system uses instead of the names that we humans use (in other words, by ''1000'' instead of ''rsgranne''; if you don't know what I'm talking about, take a look at your ''/etc/passwd'' file and run the command <tt>man</tt> <tt>5</tt> <tt>passwd</tt>), then add this preference to your profile as well:
numericids = true
numericids = true

Current revision

Ubuntu Hacks

Hacks 74-86

Yes, Ubuntu runs pretty smoothly out of the box, and if you've got a system that one person uses, you might not have to do much in the way of system configuration. However, the time may come when you need to add new users or provide tech support to friends and family you've turned onto Ubuntu. If so, then you'll find hacks in this chapter to help you out.

There are plenty of other cases where you may need to wear a system administrator's hat, and there are hacks in this chapter to help with those, too. Perhaps you need to manage and mount external drives, or mount directories from remote servers. And once in a while, things will go wrong: in many cases, you can peek into the system logs to figure out what happened, but sometimes you'll need to resort to a rescue disc.

And there's one system administration task no one can avoid: backups. You'll find hacks in this chapter for making traditional backups and keeping files between two or more computers in sync.

Contents

Edit Configuration Files

When you need to edit a configuration file from the command line in a pinch, use these tips for the ins and (especially) outs of vim.

If one thing is for sure about Linux, it's that it has no shortage of text editors. This reflects the largely text-based nature of Linux, from the command line, to the source code, to the configuration files that programs refer to. When you are in a desktop environment, you can use one of many graphical tools to edit text files; however, in an emergency situation, or when logged into a machine remotely, you may need to fall back on a command-line text editor. Under Ubuntu, the default text editor is vim, and this hack provides you with the basic information you need to make changes to configuration files using vim.

The vi editor has a rather mixed reputation. Some people love how quickly you can edit files with it, and others hate its steep learning curve. The vim editor (short for "Vi IMproved") takes vi and adds not only a number of powerful improvements, but also a number of user-friendly changes such as a complete, integrated help environment. Ubuntu includes vim by default, and even if you run vi, you are actually running vim in a vi-like emulation mode.

So, to edit a file, run vim on the command line followed by the filename as an argument. For instance, to edit the filesystem table (/etc/fstab), type:

$ sudo vim /etc/fstab
         

Unlike most other editors you may be used to, vim operates via a number of modes. Different keys perform different functions depending on which mode you are in. (This can be rather confusing the first time you use vim, since you will start typing only to see all sorts of strange and undesirable behavior.) For starters, there are two main modes you will be operating in: Normal mode (for navigating the file) and Insert mode (for editing text).

Navigate Through the File

Normal mode is the main mode you default to when you start vim. In this mode, you use a number of different keys to move through the file. Table 8-1 lists some of the main navigation keys and their functions.

Table . VIM Normal Mode keybindings

Navigation key Function
h, j, k, l Move left, up, down, or right, respectively. The arrow keys will also move the cursor.
w, b Move forward or backword one word at a time, respectively. Useful when you want to skip through a file quickly.
^, $ Move to the beginning or the end of the current line, respectively.
gg, G Move to the beginning or the end of the file, respectively.
:number Move to the specified line number in the file.
/keyword Search for the specified keyword; hit n to move to the next keyword.

Edit a Line

Once you have moved to where you wish to edit, you have a number of different options to edit a line. If you simply need to delete some text, the x key will delete the character under the current cursor. To add or change text, you will need to switch to Insert mode. Once in Insert mode, vim behaves a lot like other text editors; when you type a particular key, it will appear in the file. There are a number of ways to get into Insert mode, depending on where you want to start editing, but the simplest is to hit i, which will let you edit at the current position of the cursor.

Once in Insert mode, you can make any changes you need to a particular file. You can hit the Backspace or Delete key to delete characters and can still use the arrow keys to move around in the file. Once you are finished, hit the Esc key to exit Insert mode.

Copy and Paste

Sometimes when editing a file it is useful to copy and paste one line or a section of a line to another line. There are a number of ways to do so, some faster than others, but for new vim users, one of the easiest ways is through vim's Visual mode. To select a portion of text, type Esc until you are in Normal mode, move to the beginning of the text you want to copy, and then hit the v key to switch to Visual mode. Now move the cursor as normal, and you will notice that characters will be selected as it moves. When you have selected all of the characters you want, hit the y key to copy (or yank) the text. Then move to where you want to paste it and hit p.

Tip

If you know you want to copy full lines at a time, enter Visual mode with capital V instead of v. Now move up or down to select a number of lines and type y to copy and then p to paste your lines.

Undo

Once nice feature that vim has that vi doesn't is unlimited undos. If you make a mistake and want to undo it, hit Esc until you are in Normal mode, and then simply type the u key to step back through each edit. If you find you need to redo a change, type Ctrl-R.

Save and Exit

After you have made your changes and are ready to save, you need to switch to Command mode. Command mode lets you run special vim commands, including the commands you need to save changes and exit. To save (or write) your changes, type :w and press Enter. When you are ready to quit, type :q and press Enter. You can also combine these two commands into a single command, and type :wq and press Enter.

Search and Replace

A common need when editing files is to substitute one word for another throughout the file. To do so in vim, type:

:s/word/replacement/g

to change all instances of the word on a particular line. To change all instances of the word within a file, type:

:%s/word/replacement/g

If you want to give confirmation before making each change, add a c to the end of the command:

:%s/word/replacement/gc

and vim will ask you before it makes any changes.

One particularly handy use of the search-and-replace feature when editing configuration files is for commenting or uncommenting multiple lines in the file. To do so, enter Visual mode, select each line you want to comment, and then type:

:s/^/#/

to comment the lines out with the # character. Replace # with whatever character that file uses for comments. If you instead want to uncomment a number of lines, select them in Visual mode and then type:

:s/^#//

or replace # with the character your file uses for comments.

Help

If you find you are lost, or you just want some help with vim's options, type:

:help

to enter vim's help screen. You can also follow :help with a particular keyword, so if you want to find out what the w key does, for instance, type:

:help w

Where to Learn More

This hack is by no means an exhaustive tutorial on how to use vim. vim has a number of advanced and powerful features that, once learned, allow you to make rapid changes to files. If you want to know more about vim's features and how to use them, type:

$ vimtutor
            

at the command line to try vim's built-in interactive tutorial program.

Manage Users and Groups

Use GUI and command-line tools to add, remove, and modify users and groups on the system.

So you have used your Ubuntu desktop for some time, and it has started to attract attention from the other members of your household. They all want in on the action, but you may not necessarily want them to be able to edit your files. The solution is to create new accounts for each person that wants to access your system. That way, everyone will have their own login, password, and home directory to store files. Under Ubuntu, you have a number of options to edit users and groups, both with GUI and command-line tools.

Use the User Administration Tool

Ubuntu provides a GUI tool that makes user and group management relatively easy. Click System→Administration→Users and Groups to start the user administration tool (Figure 8-1). The interface is split into two tabs, the first for users and the second for groups, and has buttons on the right side to add, edit, and delete users and groups. By default, only the users and groups you most likely want to edit are displayed, but you can toggle the "Show all users and groups" checkbox at the bottom of the window to see everything.

Figure 8-1. Ubuntu user administration tool

Ubuntu user administration tool

To add a new user, click the Add User button. A new window will appear with a number of fields to fill out (Figure 8-2). The Username and Password fields are the most crucial to fill out. You can choose a password or you can have Ubuntu generate a random password for you. If you are a more advanced administrator, click on the Advanced tab to change the default group, shell, home directory, and user ID. Ubuntu also provides three user profiles to choose from. The Default profile has no extra privileges; the Desktop profile will give the user access to audio, video, and other peripheral devices; and the Administrator profile will allow the user to become root (despite its name, Default is not the default choice; Desktop is). All of these settings, including user access, can be changed later on; just select a particular user and click Properties. To remove a user, select it and click Delete. Note that none of these changes will be applied until you click OK and close the program.

Figure 8-2. User Account Editor configuration window

User Account Editor configuration window

The Groups tab lets you configure group membership on the system. Group membership dictates who can sudo to root (members of the admin group) and who can administer printers, scanners, and other devices. To see the members of a particular group, select it and then click Properties (Figure 8-3). You can select new users to add to the group from the list on the left side of the window. To create a new group, click the Add Group button and fill out the fields in the window that appears.

Figure 8-3. Ubuntu group account editor

Ubuntu group account editor

Manage Users and Groups from the Command Line

You can also add, change, and remove users and groups with command-line tools. To add a new user to the system, type:

$ sudo adduser 
               
                  username
               
            

You will be prompted for the user's real name, along with other optional information, and, finally, you will be asked to enter the user's password. The adduser script will automatically create the user's home directory and a group for the user. You can change the user ID, default home directory, and shell with the --uid, --home, and --shell options, respectively. Later on, to tweak the user's settings, open the /etc/passwd file with your preferred text editor as root (read "Edit Configuration Files" [Hack #74] for a primer on editing text files from the command line).

Warning

If you create a user at the command line, the new user will not be assigned to the correct groups (such as audio) needed for the full desktop experience. You can use sudo to edit the /etc/group file and give the new user the appropriate group memberships, using the first user you created as a guide.

To delete a user, use the deluser command:

$ sudo deluser 
               
                  username
               
            

By default, this will remove only the user from the system, not any of the user's files. To remove the user's home directory and mail spool, add the --remove-home option. To remove all files on the system owned by the user no matter where they are, use the --remove-all-files option.

There are similar command-line commands to add and remove groups from the system. To add a group to the system, use the addgroup command:

$ sudo addgroup 
               
                  groupname
               
            

You can use the --gid option to specify which group ID number to use. All system groups are found in the /etc/group file, so you can edit this file as root if you want to tweak group membership or other settings. If you want to add a current user to a current group, instead of editing /etc/group, you can type:

$ sudo addgroup 
               
                  username
                
               
                  groupname
               
            

To remove a group from the system, use delgroup:

$ sudo 
               delgroup 
               
                  groupname
               
            

Warning

Be careful when deleting users or groups, particularly any users or groups you did not explicitly create. Accounts created by Ubuntu when it was installed, or when certain packages were installed, should generally stay as they are. Leave them be unless you know what you are doing.

Mount Any Filesystem

Tweak /etc/fstab to control which filesystems are mounted at boot.

By default, Ubuntu will automatically detect and configure mount points for any partitions it finds when Ubuntu is installed. However, if you add a new disk to the system, or you want to automatically mount an NFS or SMB share at boot time, you must resort to the tried-and-true Linux method: editing /etc/fstab.

The /etc/fstab file (short for filesystem table) keeps track of filesystems that you want to mount in static locations. Here is a standard Ubuntu fstab file, which provides a good example for what each field in the file stands for:

# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/hda1       /               ext3    defaults,errors=remount-ro 0      1
/dev/hda5       none            swap    sw              0       0
/dev/hdc        /media/cdrom0   udf,iso9660 user,noauto 0       0
/dev/fd0        /media/floppy0  auto    rw,user,noauto  0       0

Here you see a list of partitions Ubuntu will mount, where it will mount them, what type of filesystem they use, any special options that might need to be passed to the filesystem, whether the partitions should be included in partitions the dump utility backs up, and what order filesystems are checked at reboot time. It's a lot of information, to be sure, but when you want to add a new filesystem to the mix, there are only a few fields that are crucial.

Add a Local Partition

One of the simpler partitions to add to the fstab file is a local partition. Suppose, for instance, that we added a second IDE hard drive to the system (hdb), and it had a Windows FAT32 partition as the first partition (/dev/hdb1) and a Linux EXT3 partition as its second partition (/dev/hdb2). First, we need to create a place these filesystems will mount, so we create two new directories under /mnt called windows1 and linux1:

$ sudo mkdir /mnt/windows1 /mnt/linux1
            

Now we can add two lines to our /etc/fstab file, one for each of the new partitions:

/dev/hdb1     /mnt/windows1   vfat    defaults     0       0
/dev/hdb2     /mnt/linux1     ext3    defaults     0       2

Notice the differences between these two lines. Besides the fact that the first column in each line lists a different partition and the second column has a different mount point, notice that in the third column we specified vfat for the filesystem type for the Windows partition and ext3 for Linux. The value that goes here varies depending on what type of partition you are mounting. Table 8-2 displays some of the common partition types and what value to put in /etc/fstab.

Table . Filesystems and their fstab labels

Filesystem fstab label
Windows FAT, FAT32 vfat
Windows NTFS ntfs
Linux EXT3 ext3
Linux EXT2 ext2
ReiserFS reiserfs
XFS xfs
JFS jfs
SMB Network File Share smb
NFS Network File Share nfs
CD-ROM / DVD-ROM udf,iso9660
RAM Filesystem ramfs
Compressed RAM Filesystem cramfs


In the fourth column, both fields list defaults, which mounts the filesystem with default settings. Different filesystems support different mount options, such as whether to mount the filesystem as read-only or read/write, the sorts of ownership that files in a partition might have, and so forth. We can get more information about special options that a filesystem supports from the mount manpage:

$ man 8 mount
            

In the fifth column, we left the values at 0 because we are not planning on using the dump utility to back up these partitions. This is a safe value for most partitions you might want to mount.

In the final column, notice we set the Windows partition to 0 and the Linux partition to 2. We did so because we don't want Linux to automatically run fsck on our Windows partition, yet we do want the Linux partition to be among the drives that are checked. We set the Linux partition's value to 2, because the standard is to set the root filesystem (/) to 1 so that it is checked first, then set all other Linux filesystems to 2.

Mount the Filesystems

Now that these two filesystems are configured, we can use the mount command to mount them:

$ sudo mount /mnt/windows1
$ sudo mount /mnt/linux1
            

While we could have used the mount command to mount these filesystems manually, without bothering to add any entries to fstab, when we set up the filesystems in fstab, we don't have to list a lot of options—just the device or mount point we want to mount. After mounting our devices, we can type either mount or df to see all of our mounted filesystems.

Mount Network Filesystems

Network filesystems use slightly different syntax than ordinary partitions, so they get a separate section here. Specifically, the syntax you use to describe the filesystem (the first column) is different. For this example, we have an NFS share on host file1 at /mnt/shares, and a SMB file share on host file2 called data. We want to mount the NFS share at /mnt/share1 and the SMB share at /mnt/data1. To mount both of these partitions at boot time, we would add the following lines to our /etc/fstab:

file1:/mnt/shares     /mnt/share1   nfs    defaults     0       0
//file2/data          /mnt/data1    smb    defaults     0       0

Notice the difference in syntax. NFS shares follow the hostname:/path/to/share syntax, while SMB shares follow the //hostname/share syntax. Other than that, the remaining fields are similar to the first examples, and we could pass special NFS or SMB options in the options field if we needed to.

Tip

If you don't want a partition to mount at boot time (like floppies and CD-ROM drives), add the noauto option to the list of options.

To mount these shares, we would make sure that the /mnt/share1 and /mnt/data1 directories existed, and then we would type:

$ sudo mount /mnt/share1
$ sudo mount /mnt/data1
            

As with local partitions, we can use the mount or df commands to check the status of these mount points.

For more information on the syntax for the /etc/fstab file, and what sorts of mount options you can use for particular filesystems as well as what filesystems Linux can mount, see the fstab(5) and mount(8) manpages, respectively.

Control Startup Services

Use GUI and command-line tools to start, stop, disable, and enable services.

As Ubuntu boots, you might notice text scrolling by, detailing all of the different things Ubuntu is doing. Among these things are a number of services that Ubuntu enables at boot time, such as the cron scheduling service, the system logger, and the graphical login manager. If you have installed other services on your system, such as a web server, those services will also be enabled at boot. Sometimes, though, you may want to either stop or temporarily disable these services, and Ubuntu provides a number of ways to do this, both graphically and through the command line. This hack shows some of the more common ways to control startup services.

Services Administration Tool

Ubuntu provides a graphical tool to manage services that start up at boot time. Click System→Administration→Services, or type:

$ sudo services-admin
            

to start the Services Administration Tool. Figure 8-4 shows how simple the program is: you have a list of services with a checkbox next to each of them. To disable a service, just deselect its checkbox and click OK. Currently, this application supports changing only whether a service starts at boot, so to manually start, stop, or restart a service, you will need to refer to the command-line method.

Figure 8-4. Ubuntu Services Administration Tool

Ubuntu Services Administration Tool

Command-Line Method

Before learning how to start, stop, and disable services from the command line, it's important to understand Ubuntu's startup process and how Ubuntu determines which programs to run when it starts. For most Linux distributions (including Ubuntu), System V init scripts govern which programs start at boot and which programs don't. All System V init scripts that could potentially be run at boot are typically located in the /etc/init.d/ directory for Ubuntu. Not every script in /etc/init.d is executed at boot, however. Linux organizes which scripts to run for different circumstances into runlevels; most Linux systems have seven runlevels, ranging from 0 to 6. Think of a runlevel as a checklist of programs for Ubuntu to start before it presents a login.

A few of these runlevels are set aside for special states in a Linux system:

Runlevel 0
Halts the system.
Runlevel 1
Sets up single-user mode.
Runlevels 2-5
Set up different multiuser modes. Although, typically, only one or two of these are used by a distribution.
Runlevel 6
Reboots the system.

Each runlevel has a directory that stores symlinks to certain init scripts in /etc/init.d, which are started when that runlevel is selected and stopped when it is exited. Ubuntu puts these symlinks under /etc/rc<runlevel>.d—for example, all runlevel 2 scripts are located in /etc/rc2.d/.

If you look in one of these runlevel directories, you'll notice that many of the symlinks to scripts in /etc/init.d have odd names that begin with an S, K, or D; then a number; and finally the name of the script. Ubuntu defaults to runlevel 2, so here is a sample /etc/rc2.d directory:

greenfly@ubuntu:~$ ls -l /etc/rc2.d/
total 0
lrwxrwxrwx 1 root root 20 2006-01-21 14:48 K77ntp-server -> ../init.d/ntp-serverlrwxrwxrwx 1 root root 17 2006-01-07 08:46 S05vbesave -> ../init.d/vbesave
lrwxrwxrwx 1 root root 15 2006-01-07 08:19 S10acpid -> ../init.d/acpid
lrwxrwxrwx 1 root root 18 2006-01-07 08:17 S10sysklogd -> ../init.d/sysklogd
lrwxrwxrwx 1 root root 15 2006-01-07 08:17 S11klogd -> ../init.d/klogd
lrwxrwxrwx 1 root root 14 2006-01-07 08:47 S12dbus -> ../init.d/dbus
lrwxrwxrwx 1 root root 13 2006-01-07 08:50 S13gdm -> ../init.d/gdm
lrwxrwxrwx 1 root root 13 2006-01-07 08:46 S14ppp -> ../init.d/ppp
lrwxrwxrwx 1 root root 16 2006-01-07 08:48 S19cupsys -> ../init.d/cupsys
lrwxrwxrwx 1 root root 15 2006-01-07 08:52 S19hplip -> ../init.d/hplip
lrwxrwxrwx 1 root root 14 2006-01-07 08:47 S20apmd -> ../init.d/apmd
lrwxrwxrwx 1 root root 22 2006-01-07 08:48 S20hotkey-setup -> ../init.d/hotkey-setup
lrwxrwxrwx 1 root root 21 2006-01-07 08:46 S20laptop-mode -> ../init.d/laptop-mode
lrwxrwxrwx 1 root root 17 2006-01-07 08:16 S20makedev -> ../init.d/makedev
lrwxrwxrwx 1 root root 23 2006-01-07 08:18 S20nvidia-kernel -> ../init.d/nvidia-kernel
lrwxrwxrwx 1 root root 19 2006-01-07 08:48 S20powernowd -> ../init.d/powernowd
lrwxrwxrwx 1 root root 15 2006-01-07 08:46 S20rsync -> ../init.d/rsync
lrwxrwxrwx 1 root root 21 2006-01-07 08:47 S25bluez-utils -> ../init.d/bluez-utils
lrwxrwxrwx 1 root root 15 2006-01-07 08:17 S25mdadm -> ../init.d/mdadm
lrwxrwxrwx 1 root root 17 2006-01-07 08:46 S89anacron -> ../init.d/anacron
lrwxrwxrwx 1 root root 13 2006-01-07 08:46 S89atd -> ../init.d/atd
lrwxrwxrwx 1 root root 14 2006-01-07 08:46 S89cron -> ../init.d/cron
lrwxrwxrwx 1 root root 17 2006-01-07 08:18 S98usplash -> ../init.d/usplash
lrwxrwxrwx 1 root root 22 2006-01-07 08:46 S99acpi-support -> ../init.d/acpi-support
lrwxrwxrwx 1 root root 19 2006-01-07 08:16 S99rmnologin -> ../init.d/rmnologin
lrwxrwxrwx 1 root root 23 2006-01-07 08:16 S99stop-bootlogd -> ../init.d/stop-bootlogd

As you can see, this directory is full of symlinks that point to a script in the init.d directory. The letter at the beginning of each filename tells init when to execute this script. If the script begins with an S, then init starts the script when it goes through the runlevel. If the script begins with a K, then init stops (or kills) the script when it changes to a different runlevel. If the script begins with a D, then that script is disabled for the time being, and init ignores it. init runs the scripts in numerical order, so the numbers in each script let you know in which order they are to be run. This is useful to ensure that dependent services start after the service they are dependent on.

When Linux boots and starts the init process, it reads its configuration from /etc/inittab, which configures each available runlevel, the default runlevel to use, and some other settings. Next, init loads any system scripts from a special system runlevel directory at /etc/rcS.d. These scripts load daemons and services that are vital to the boot process. Lastly, init runs any startup scripts for the default runlevel in alphabetical order.

Warning

Scripts in /etc/rcS.d are run in runlevels 1 through 5, so you should generally leave them alone unless you know what you are doing. If you accidentally disable a crucial service, you may have to resort to a rescue disc to undo the mistake.

Change the Runlevel

You can change the runlevel yourself on the command line with the init command. To switch to single-user mode from the command line, type:

$ sudo init 1
            

Warning

If you're running X11 when you issue this command, beware, since it will kill X and your desktop environment!

This command runs all of the shutdown scripts for your current runlevel and then any startup scripts for single-user mode. To change back to the default multiuser runlevel for Ubuntu, type:

$ sudo init 2
            

Tip

You can also use init to halt or reboot a machine: just change to runlevel 0 and runlevel 6, respectively.

Manually Start and Stop Services

You can start and stop scripts manually by running the script with the start or stop argument. For example, to stop the CUPS service from the command line, type:

$ sudo /etc/init.d/cupsys stop
            

To start the service back up, type:

$ sudo /etc/init.d/cupsys start
            

Most scripts also support a restart argument that will run stop, then start for you. Most init scripts are also configured to output the list of arguments they support when you execute them without any options:

greenfly@ubuntu:~$ sudo /etc/init.d/cupsys
Password:
Usage: /etc/init.d/cupsys {start|stop|restart|force-reload|status}

Disable Scripts from Starting

To disable a script, you must know your default runlevel. On Ubuntu, the default runlevel is usually set to 2, but you may want to double-check your default runlevel before you start disabling services. All runlevels are defined in /etc/inittab, so to check the default runlevel, type:

greenfly@ubuntu:~$ grep initdefault /etc/inittab
id:2:initdefault:

As you can see, in this example, the default runlevel is in fact 2. Now change to the directory containing all of the scripts for that runlevel (/etc/rc2.d) and find the script you want to disable. To disable a service, just rename its script by changing the S to a D. For example, to disable the cupsys service, type:

greenfly@ubuntu:~$ cd /etc/rc2.d
greenfly@ubuntu:/etc/rc2.d$ sudo mv S19cupsys D19cupsys
            

To enable it again, rename it back by changing the D to an S:

greenfly@ubuntu:~$ cd /etc/rc2.d
greenfly@ubuntu:/etc/rc2.d$ sudo mv D19cupsys S19cupsys
            

You'll still need to stop the service as shown earlier if you want to shut it down right away, but renaming it will control whether it's started the next time you reboot (or change runlevels).

Build Kernels the Ubuntu Way

Special tools take the pain out of compiling and installing custom kernels.

Debian boasts some very useful tools to make the job of building and installing kernels much easier, and of course those same tools are available in Ubuntu as well. They streamline the process of compiling a custom kernel and building a .deb package around it, allowing you to install a new kernel the same way you would any other package. This makes it possible to build a kernel on one machine and then just install the package on other machines, without having to recompile or track down all the loose ends—great if you need to upgrade a number of similar machines!

To build a custom kernel "the Ubuntu way," you will need to get hold of the kernel source and a variety of tools to configure, compile, and package it.

Get the Source

You can get the official kernel source yourself directly from http://ftp.kernel.org/pub/linux/kernel/, or a mirror if you like, but of course there are also packages of the kernel source. A quick:

$ apt-cache search linux-source
            

will get you a list of Linux kernel source packages all ready to go. Picking one as an example, you could just run:

$ sudo apt-get install linux-source-2.6.15
            

to download the source to your system. You'll then end up with an archive sitting in /usr/src, which is where most kernel work is done. Now you can extract the source package:

$ cd /usr/src
$ sudo tar -xjf linux-source-2.6.15.tar.bz2
            

That will leave you with a /usr/src/linux-source-2.6.15 directory, decompressed and ready to configure. First, though, create a symlink to it called linux, like this:

$ sudo ln -sf linux-source-2.6.15 linux
            

The f option forces creation of the new symlink even if there's already an existing link to an old kernel source. Now you can easily get to your kernel source tree just by typing:

$ cd /usr/src/linux
            

Tools You Will Need

To install the main tools, you will need to run:

$ sudo apt-get install kernel-package libncurses5-dev
            

This will also cause your computer to pull down a big list of supporting packages that are dependencies.

Configuration Methods

There are a number of ways to configure a kernel prior to compiling it, and all but the first are invoked as arguments to the make command. Make sure you have a shell open in the kernel source directory (which should be /usr/src/linux if you followed our directions earlier) before you try any of these. In increasing order of sophistication, they are:

editing .config
The actual configuration is saved in the source directory as a text file called .config. If you really want to (or if you are trying to find an option some README has listed by name, such as CONFIG_PACKET) you can edit this file directly using a text editor such as vim or Anjuta. This should need to be done very rarely, though.
make config
The most basic approach, this will just ask you a whole heap of questions, one after another. Personally, we hate configuring the kernel this way, because it's sequential and you have to go through a heap of stuff you probably couldn't care less about. Only bother with this as a last resort, such as if other options aren't available to you for some reason.
make menuconfig
This is the most common way to do the configuration. It displays a nice, keyboard-driven menu that you can navigate using arrows, Enter, and the spacebar. However, one catch that can trip you up is that you need to have the development libraries for Ncurses installed, not just Ncurses itself. That's why we had you install the libncurses5-dev package a few paragraphs ago!

make xconfig
The preferred method for machines with X, this option is very similar to menuconfig, except it's all done in a nice point-and-click GUI with mouse navigation. The possible gotcha to getting this working is you'll need tk8.3. If you get errors saying it can't find "wish," run:

$ sudo apt-get install tk8.3
                     

and all should be well.


Configuring the Kernel

Whichever configuration method you choose, the next thing to do is to examine the settings for the different kernel options and make changes as required.

There are a huge number of kernel options, so to help you find things, they are grouped in a logical way in different major and minor sections. The different configuration methods have their own ways to represent this, but they should all be fairly straightforward to follow.

We won't go into details of how to do the kernel configuration here, because what you'll need to configure will vary dramatically, and there are plenty of tutorials on the Net, including the Kernel-HowTo, that explain this in detail. Basically, though, you need to work through each menu in turn to find options and modules you want to enable, and set them to either off (not available), on (compiled straight into the kernel), or module (compiled separately so it can be loaded when needed).

Once you've finished going through the options, quit and save to have your choices written out to a configuration file to be used by the compiler.

One little trick to note is that the actual configuration details are stored in a file called .config in the kernel source directory. Once you've gone to all the trouble of setting up a kernel the way you want it, moving to a new kernel can be painful if you have to go through the configuration process from scratch, and you'd be almost certain to miss something crucial—we know we usually do! To make things easier for yourself, you can copy the .config file into the new kernel source directory to have all your options carried over automatically. Then you can run the configuration again just to check things over, save, and go on as before. Provided you're not changing to a totally new kernel type, this trick can save you a lot of time. Beware if you're moving to a totally different kernel, though, such as from 2.4.x to 2.6.x, because many of the options will be totally different and while the old config will still work, it may not have some options activated that appear only in the new kernel. You can also run make oldconfig after copying in the old .config file, which will then prompt you only for options that are new or have changed since you configured your previous kernel.

Compiling and Packaging

This is the point when your kernel is actually compiled and placed in a package. Normally, that would be a lot of work, but thanks to the tools provided by kernel-package, it's now one of the easiest steps.

If you have read a traditional Kernel HOWTO, you've probably seen a sequence of commands such as make dep && make clean && make bzImage used to build the dependencies and then the kernel itself. You don't need to do any of that; instead, just type:

$ sudo make-kpkg kernel-image
            

while in the /usr/src/linux directory, and those steps will all be taken care of. The kernel-packaging tools first compile your kernel and modules according to the configuration you just generated and then build a Debian package called kernel-image-<version_architecture>.deb outside the current source directory—i.e., in /usr/src.

This stage can take a while depending on your machine speed and which modules you selected. Expect anything from 10 minutes to a half hour, during which time you'll see a very long stream of debugging information that probably won't be of any interest to you at all, unless something goes seriously wrong.

Install Your Kernel Package

You're almost there! What you have now is a kernel, custom-compiled according to your requirements and set up as a Debian package ready to install.

Installing your new kernel is now just a matter of using dpkg to install the package as you would any other Debian package:

$ sudo dpkg -i 
               
                  kernel-image-2.6.15_10.00.Custom_i386.deb
               
            

(replacing kernel-image-2.6.15_10.00.Custom_i386.deb with whatever your package is called). It may ask you if you want to create a boot floppy using that image; the dozens of machines we manage don't have a single floppy drive among them, so we always say no, but if you want to, you can say yes to have it create one for you.

dpkg will also take care of updating your bootloader configuration so your new kernel will be available next time you boot. Both GRUB and LILO are managed automatically, but if you use a different bootloader, you may need to update it yourself at this point.

What happens when you install the package is that dpkg puts your new kernel image in the /boot directory where kernels are normally stored, creates a symlink from /vmlinuz to your new kernel (you can verify this by typing ls -l /vmlinuz), modifies your /boot/grub/menu.lst or /etc/lilo.conf as appropriate so your bootloader can find your new kernel, and moves and renames the previous kernel so it's available as an emergency fallback in case your new kernel borks.

Note that at this point, you will not actually be running the new kernel; it's just set up, ready to go for next time you reboot. Installing a new kernel is one of the very few things that actually requires you to reboot a Linux system.

Rebooting and Testing

Time for the big test! If you are running X, exit the session and reboot. If not, just type sudo reboot. When GRUB loads, you will have the option of pressing Esc to see a list of available kernels, including your new one. Select it and hit Enter.

Once your machine has booted, you can use uname to check which kernel you are running:

$ uname -a
            

For more information about which modules were loaded and whether the new kernel correctly detected your hardware, you can look through the dmesg log:

$ dmesg | less
            

If everything worked as expected, congratulations! You've just compiled and installed a custom kernel "the [Debian|Ubuntu] way."

Installing on Other Machines

This is where the convenience of building kernels as packages becomes most obvious. If you want to install your custom kernel on other machines, the process is very simple: just copy the .deb package you created to the target machine and install it using dpkg exactly as before. Simple!

There's no need to compile the kernel on each machine, copy the source code to them, or even have a compiler installed on them. All the hard work was done once on one machine and doesn't need to be repeated.

If you run a server farm or computer lab and have a lot of local machines to install your kernel on, the best way to do so is probably to put the kernel package on a local package repository and have all your computers fetch it over the network. You can learn more about how to do so in "Create Your Own Package Repository" [Hack #65].

Install Multiple Copies of One Kernel Version

Often you'll want to install multiple copies of the same kernel version while doing testing, but that can cause problems: modules are installed in a directory based on the kernel version number, so if you want to try out different configuration options, you'll run into problems. Luckily, the kernel package tools allow you to work around that limitation by appending an arbitrary version string to the regular version string. This means you can install multiple builds of the same kernel version and keep everything neatly organized on disk, as well as select which one you want to use at startup.

To modify the version string, just pass the --append-to-version option when creating the package, like so:

$ fakeroot make-kpkg --append_to_version=-jon17 
               
                  kernel-package
               
            

This would generate a kernel package something like kernel-image-2.6.15-jon17_10.00.Custom_i386.deb.

Back Up Your System

Use Ubuntu's Simple Backup utility to easily set up standalone or recurring backups of your important files.

Backups always seem to be the thing we think about after it's too late. Just about every computer user we know has been bitten at least once by failing to make a backup of an important file or directory. Yet what should you back up? How often? Out of the hundreds of backup tools out there, which should you use? Ubuntu has answered all of these questions with the aptly named Simple Backup tool. This tool allows you to schedule automatic recurring backups of predefined important files on the system, or, if you want more control, it allows you to set fine-grained backup options that better suit your needs. In this hack, we discuss some of the main options in the Simple Backup tool and how to set up a backup-and-restore solution for your computer.

Get Simple Backup

The first step is to install Simple Backup. Its package is called sbackup and is available from the universe repository, so if you haven't yet enabled universe, you will need to do so. If you need more instructions on how to enable repositories, refer to "Modify the List of Package Repositories" [Hack #60] or the individual hacks corresponding to your preferred package-management tool [Hacks #54, #55, and #56].

Once the program is installed, two new menu entries will appear under System→Administration: one called Simple Backup Config and a second called Simple Backup Restore. Select Simple Backup Config to see the fairly straightforward main window shown in Figure 8-5.

Figure 8-5. The main Backup Properties window

The main Backup Properties window

Configure Simple Backup

The General tab displays three main options for you to choose from:

Use recommended backup settings
If you are new to Ubuntu or aren't quite sure what you should back up, but you do want to set up recurring backups, select this option. It will automatically set up daily incremental and weekly full backups of important directories on your system and store them under /var/backup. Note, however, to prevent the backup from filling up your disk, this option excludes files larger than 100 MB as well as a number of multimedia files, so if you want to back up either of those, move to the next options.
Use custom backup settings
If you have specific backup needs, or the default choices from the recommended backup don't suit you, choose this option to access the fine-grained configuration options in the following tabs (more on them shortly).
Manual backups only
If you don't want to set up a recurring backup schedule, but you do want to be able to manually back up some files from time to time, select this option and read on for more information on how to configure what to back up in the following tabs.

Use recommended backup settings

If you are new to Ubuntu, don't know what exactly to back up, or don't want to fuss with a configuration, check "Use recommended backup settings" in the General tab and then click Save. Ubuntu will schedule a nightly incremental backup of important files on the system, along with a weekly full backup. If you later find you need to restore a file, see the "Restore from Backup" section later in this hack.

Use custom backup settings

There are a number of reasons why the default backup settings might not suit you. For instance:

  • You may have a number of important files that are larger than 100 MB, and you need to be able to back those up.
  • You want to keep a backup of your multimedia files.
  • You want to save the backups somewhere other than /var/backup (either another directory on the system or possibly a directory on a remote system).
  • You want to include or exclude other files or directories from the backup.
  • You want to schedule your backups to run at different times or different frequencies from the default.

If you want to enable custom recurring backups, select "Use custom backup settings." If you want to configure a custom manual backup instead, select "Manual backups only." Apart from the Time tab being disabled in the manual backup option, the configuration ability will be identical.

Configuring Custom Backup Settings

Here are some settings you can modify to customize the backup of your system.

Configure files to include

By default, Ubuntu includes the major important directories you might want to back up on a system, such as /etc, /var, /home, and /usr/local. It's possible you might have an extra directory or file that you want included in addition to these (possibly an extra mount point under /mnt). If so, click the Include tab to see the directories and files currently included in the backup (Figure 8-6).

Figure 8-6. Define which directories to include in the backup

Define which directories to include in the backup

Configure files to exclude

There are a number of different files that Simple Backup excludes by default. Click the Exclude tab to configure these exclusion options. Figure 8-7 shows an example of some multimedia file types that are excluded by default. If you do want to back up your MP3s for instance, select "mp3" from the list of excluded files and click Remove. If you want to add a particular file type to the exclusion list, click the Add button and choose a predefined file type from the drop-down menu, or enter the file's extension and then click OK. This particular exclusion list depends on actual filename extensions, so it will work only on files with the correct extensions in their filenames.

Figure 8-7. Define which file types to exclude

Define which file types to exclude

A number of different exclusion properties are allowed via the tabs along the left side of the window. In addition to file types, click Paths to exclude by individual files or directory paths. If you are familiar with defining patterns using regular expressions, click the Regex tab and you will be able to define an exact pattern to match files to exclude. The final tab in this window, "Max size," is important because it will greatly influence the size of your backups. If you choose the custom configuration, it will default to excluding files larger than 10 MB. Depending on the size of the files you want to back up, you may want to raise or completely disable this option, but note that in doing so you risk filling up your backup directory if it doesn't have adequate space.

Configure the backup destination

The Destination tab allows you to change where Simple Backup will store backups. You can select the default of /var/backup, choose a custom directory somewhere on your system, or even select a remote FTP or SSH share. To back up to a remote share, use the "Connect to Server" dialog shown in Figure 8-8 or type ssh:// or ftp:// followed by the username:password, then @, then the remote host to connect to and the remote directory.

Figure 8-8. Back up to a remote share

Back up to a remote share

Configure recurring backups

If you want to have Simple Backup automatically back up your system periodically, click the Time tab to configure how frequently to back up (Figure 8-9). You can choose anything from never backing up to backing up hourly, daily, weekly, or monthly. You can also configure the time, day of week, or day of month to back up, depending on how frequently you want to back up the system. Finally, you can configure how often to do a full backup. By default, a full backup is performed weekly, with incremental backups being performed otherwise to save space.

Figure 8-9. Schedule your backups in the Time tab

Schedule your backups in the Time tab

When you are finished with your backup configuration, click Save to save the configuration and exit, or click Backup Now to start the backup right this instant based on your configuration.

Figure 8-10. The "Restore files/directories" window

The "Restore files/directories" window

Restore from Backup

So you have accidentally deleted a file, or you need to roll back to a previous version of it. If you have already configured recurring backups, or have manually performed backups using Simple Backup, click System→Administration→Simple Backup Restore. Figure 8-10 shows the main window with a particular backup set selected. You can choose from any previous backups from the "Available backups" drop-down menu, and you can then browse through the directory structure of the backup to find the file or directories you want to restore. Once you have found the files or directories you want to restore, click Restore to overwrite any current version of the file with the backed-up version, or click Restore As to restore the file or directory to a new location with a new name so you don't overwrite the current version.

Clone an Installation

Export a list of installed packages on one Ubuntu system, and import them into another to build a duplicate system.

We've installed many different Debian-based distributions over the years, and one thing we've found handy to have is a complete list of packages you have installed. If you want to create a system that is similar to a different system you have already set up, it can be difficult to remember each and every package you had installed. In this hack, we cover a method to export your current package list and import it into a new system.

Tip

This method works best when you are exporting to and importing from the same distribution and, specifically, the same release—for example, exporting from Ubuntu Dapper to Ubuntu Dapper. Because of the differences in package versions and dependencies across releases, and especially across distributions, you will have substantially more headaches with conflicting packages if you try to export, say, from Ubuntu Breezy to Ubuntu Dapper or, worse, vice versa.

Export the List of Installed Packages

The first step in cloning an installation is to grab the complete list of installed packages from the first system. To do so, you basically instruct dpkg to dump its entire list of packages, filter out any packages that aren't currently installed, and then redirect the output to a file:

$ sudo dpkg --get-selections | grep '[[:space:]]install$' | \\
                 awk '{print $1}' > package_list
            

Next, copy this text file to the destination system over the network, via a USB key or whatever method you prefer. You may also want to copy over the /etc/apt/sources.list file from the base system, since the new system may not have all of the same repositories enabled (if the repositories aren't the same, the destination system may not be able to find some of the packages in the list).

Prepare the Destination System

Now you need to prepare the destination system. If both systems are running the same release of Ubuntu, this may be as simple as just copying the initial system's /etc/apt/sources.list on top of the one on the current system. Otherwise, if the systems are from different releases, you will want to compare the initial system's sources.list with the /etc/apt/sources.list file on your destination system and see if there are any extra third-party repositories or repository subcategories that need to be enabled. (Read "Modify the List of Package Repositories" [Hack #60] for more information on Ubuntu's repositories and how to edit sources.list.)

Once your sources.list file is settled, update your package list to make sure you get the latest version of the packages:

$ sudo apt-get update
            

Import the Package List

To import the package list, pipe the entire list to xargs, which then splits it into manageable chunks for the apt-get command:

$ cat package_list | xargs sudo apt-get install
            

If you are migrating to a different Ubuntu release, this will require a bit of trial and error, since you will likely get complaints about packages no longer existing. The simple fix for this is to edit the package list and remove the package from the list, save your changes, and then rerun the command. You will likely need to do this a number of times, but eventually you will have a list of valid packages. A more complicated but thorough fix would involve checking the latest list of packages for a potential update or replacement for the package that no longer exists and installing them one by one.

Once apt-get completes, you are finished. All of the files from the package list will have been imported into the new system. Now keep in mind that this doesn't mean that all of the settings have transferred over. To do that, you will likely need to copy settings from the /etc directory or possibly other directories, depending on the package.

Hacking the Hack

It can be rather handy to have a complete list of installed packages for backup and restore purposes. An easy way to automatically generate a complete list is to have a cron job update the list of installed packages every night. To do so, create the following script and save it in /etc/cron.daily/package_list:

$!/bin/sh

dpkg --get-selections | grep '[[:space:]]install$' | \\
  awk '{print $1}' > /etc/package_list

Now make the script executable:

$ sudo chmod a+x /etc/cron.daily/package_list
            

This /etc/package_list file can then be backed up [Hack #79] with the rest of your system settings.

Rescue an Unbootable System

When you've had a little too much fun with your new Ubuntu system and it no longer boots, here are a few options to get your computer back in running order.

If you are reading this hack, we want to offer you our condolences. It can be stressful and upsetting when your system won't boot. We're sure you are thinking right now about all of those files you wished you backed up (We know this is a bad time, but once your system is back up and running, you might want to check out "Back Up Your System" [Hack #79]). We've certainly been in the same situation far too many times, but so far we've also been able to bring our systems back to life. While it would be impossible to cover every possible scenario that might cause a system to no longer boot, we're going to go over how to use the Ubuntu install CD in rescue mode to fix the problem and describe some common rescue scenarios.

Boot into Rescue Mode

First, find your trusty Ubuntu install CD and reboot your system to its initial boot screen. Among the number of options is "Recover a broken system." Select this option, and Ubuntu will start what appears at first to be the default installation program. You will be prompted for some standard language and network settings as you are during the installation, but these steps are only to set up the initial rescue environment. Notice that in the top-left corner of the screen, "Rescue mode" appears.

Continue through these dialogs until you are prompted to choose your root device. In Figure 8-11, you see a sample of this dialog with a lot of different drives. How do you pick the right one? If you aren't too familiar with the partition layout of your drives, it might require a bit of trial and error, but here's a basic rule of thumb:

Figure 8-11. The rescue-disk root-device dialog

The rescue-disk root-device dialog

  • If you installed Ubuntu as a standalone system and had Ubuntu overwrite everything on the current drive, your root filesystem is probably the very first partition on the list.
  • If you installed Ubuntu in a dual-boot configuration with Windows, your root filesystem is probably the second partition in the list.

After you choose your root filesystem, the Ubuntu CD will attempt to mount it, and it will continue to the next dialog if it was successful. If it wasn't successful, you might have chosen the wrong partition, so pick another and try again.

Common Rescue Scenarios

The next dialog will present you with a number of different rescue options (see Figure 8-12). The operation you choose here will vary greatly depending on the symptoms of your unbootable system, so instead of describing everything possible with these options, we're going to describe a few common rescue scenarios and what to do from this dialog to fix the problem.

Figure 8-12. The rescue-mode chooser

The rescue-mode chooser

The system won't go past the boot menu, or the menu is missing

This is a common symptom for an unbootable system. Perhaps you reinstalled Windows or some other operating system as a dual-boot system. Whatever the reason, your default GRUB boot menu is now missing and needs to be restored.

From the rescue operations menu, select the "Reinstall GRUB boot loader" option. The next screen that appears is the same one you might remember being prompted with when you first installed Ubuntu. It asks you where you would like to install the GRUB boot loader. Unless you remember setting up GRUB somewhere in particular, chances are you probably installed it to the Master Boot Record on your first hard drive. If so, type (hd0) and then continue. The rescue mode will reinstall GRUB and drop you back to the main "Rescue operations" menu. Select "Reboot the system," and you should hopefully be presented with your standard boot menu.

The system goes past the boot menu but can't access the root filesystem

This can be a tricky symptom, as a number of different problems can be the cause, from corrupted filesystems to new problematic kernels. Before you go through any steps in the rescue mode, first try to boot and select a different kernel option in the boot menu. If that kernel boots, then there is likely some sort of hardware support problem for your latest kernel. Check for updates to your kernel package and, in the meantime, boot from the working kernel. If you experience the same problem with the different kernels in your boot menu, continue to the following steps.

For this sort of problem, the troubleshooting will need to be done directly from the rescue CD, so select the "Execute a shell in the installer environment" option. You will then be dropped to a basic BusyBox shell where you can perform some basic diagnostics and repairs:


Test the root filesystem
First, test and see if you can read your default root filesystem. The rescue-mode CD mounted this filesystem at /target earlier in the boot process. Type:

~ # ls /target
                        

and see if you get a list of files somewhat similar to what you see in Figure 8-13 has sample df output you can reference. On a default Ubuntu install, your root filesystem will likely be on /dev/discs/disc0/part1.


Figure 8-13. Some sample diagnostics from the BusyBox shell

Some sample diagnostics from the BusyBox shell


Repair the filesystem
Now use the fsck tool to attempt to repair your filesystem. First, you will need to unmount the /target partition if it is currently mounted:

~ # umount /target
                        

The rescue-mode environment has fsck utilities only for ext2, ext3, and jfs, so if you chose another filesystem during the install, you will have to resort to a different rescue disk. Ubuntu defaults to ext3, so if you went with the default, type the following to check and repair your drive:

~ # /sbin/
                           
                              fsck.ext3
                            
                           -f 
                           
                              /dev/discs/disc0/part1
                           
                        

Replace fsck.ext3 with fsck.ext2 or fsck.jfs if you have an ext2 or jfs partition, respectively, and replace the partition with your particular root partition. Once the fsck completes, attempt to mount the partition under /target again and try to access it with ls:

~ # mount 
                           
                              /dev/discs/disc0/part1
                           
                            /target
~ # ls /target
                        

If you see a number of files and directories, congratulations, your drive is repaired! Type exit to leave the BusyBox shell and reboot, and see if you can boot into your system successsfully.


The system boots and continues through part of the boot process, but hangs at a particular service

Whether it is due to a poorly written init script or some other bug, on some rare occaisions, an init script might stall and refuse to continue through the boot process. When this happens, you can't drop to a shell to repair the problem, so you must resort to a rescue CD. Make a note of the service that is stalling, and boot into the rescue mode of your Ubuntu install CD.

To fix this sort of issue, you will need to actually drop to a shell within your root filesystem's environment, so select "Execute a shell in /dev/discs/disc0/part1" from the Ubuntu rescue operations menu. (This option might be named something slightly different if you picked a different partition as your root filesystem.)

This option will drop you to a shell from within your own root filesystem. You can navigate this filesystem and run commands as though you had actually booted into it. At this point, you will want to identify the init service that is causing you problems and disable it. "Control Startup Services" [Hack #77] has instructions on how to identify and disable startup services from the command line, so read through that hack for the specifics. In a default Ubuntu install, you will likely want to look for your init service in either /etc/rc2.d or /etc/rcS.d. Once you find the offending init script, rename it, changing the S at the beginning of the filename to a D.

Once the init script has been disabled, type exit to exit out of the shell and then reboot and attempt to boot back into your Ubuntu system. The disabled init script should not stall you this time, so you should be able to get to your default login screen.

Conclusion

It's true that these are only a few of the possible problems that could render a system unbootable. If you have tried all of these rescue options and still can't fix your system, don't give up! Read "Get Help" [Hack #2] for some information on how to enlist the help of some fellow Ubuntu users.

Check the Captain's Log

Find out where Ubuntu logs important system information so you can track down the cause of startup and system errors.

When you use a system long enough, eventually you find you need to put on your detective hat. Maybe you've added a piece of hardware or plugged in a new device, and you aren't sure whether the system is recognizing it. Maybe you upgraded some software, and now it's not working quite right. Whatever the reason, when you want to track down what's really going on under the hood of your Ubuntu system, it's time to look through logs.

A normal desktop system generates a surprising amount of logs in a day, even if nothing is wrong. When you connect to a network, plug in a new device, log in, or do any number of things, the system generates logs. The majority of system logs reside in the /var/log directory. Some of these logs overlap; for instance, logs from daemons will show up both in the daemon.log and in syslog. Here are some of the main logs you will find under /var/log, along with their uses:

syslog
syslog is the primary system log and contains log output from daemons and other programs running on the system, such as dhclient, cron, init, xscreensaver, and some kernel logs. This log is the first place to look when trying to track down general system errors.
dmesg
This log traditionally lists all of the boot-time kernel logging for a system, along with any other kernel logs related to devices and module loading. Check here to see what sorts of devices the kernel detected at boot time, as well as to track down any errors the kernel might have had when loading a module.
kern.log
Like dmesg, this log contains kernel log output; however, it has the advantage of being timestamped.
Xorg.?.log
The Xorg logs contain very detailed output from Xorg and are numbered according to which display it is running. The default Xorg session runs on display 0, so for the most part you would look through Xorg.0.log to trace down errors in Xorg.
messages
This log contains some kernel log messages along with log output from certain system programs. For instance, gconfd and some other programs log here.
daemon.log
Here, you will find the same daemon output that you would normally find in syslog, but without log output from kernels or other systems that log to syslog.
auth.log
Inside auth.log, you will find information about user authentication including every time a user logs in to the system or executes sudo.
mail.log
If you use your system as an email server, this log will contain information about incoming and outgoing messages, along with any errors.
apache/
If you have installed the Apache web server on your system, /var/log/apache contains access_log, error_log, and all of the other primary Apache logs.
cups/
CUPS is the printing system for Ubuntu. This directory contains all of the different logs for the CUPS service, so look here to debug any issues with printing on the system.
gdm/
GDM is the graphical login manager for GNOME. If you notice any problems with the main login screen, look at the logs in this directory for clues.

View the Logs

Now that you have an idea where to look for information, there are a number of different methods you can use to actually look through the logs. Ubuntu provides a nice graphical tool for log viewing called the System Log Viewer. To start this program, click System→Administration→System Log. The default window displays a sidebar to the left listing various system logs you have open and a main window listing the contents of the selected log. By default, the System Log Viewer lists only syslog, but you can click Log→Open to add a logfile to the sidebar (Figure 8-14).

Figure 8-14. Default System Log Viewer window

Default System Log Viewer window

A nice feature of the System Log Viewer is that it splits logs up according to date. Along the bottom-lefthand side of the window is a calendar with days displayed in bold if the current log contains entries for that day. Select a day from the calendar, and it will appear in the main window. You can also collapse a particular day from view in the main window by clicking the arrow next to the date.

You can also view logs the old-fashioned way from a terminal. The program less is a great, no-frills way to open up a logfile and page through it. Just type:

$ less /var/log/syslog
            

to open up the syslog (you can also replace that file with the path of the file you wish to open). You can use the arrow keys to navigate up and down the file, or you can hit G to scroll down to the very bottom of the file, or g to scroll to the top. To search within the logfile for a keyword, type / followed by the keyword to find. The keyword will be highlighted within the terminal, and you can type n to advance to the next match, or N to go back to the previous match. Hit F to have less continue to update the log as new lines are appended (a lot like running tail -f on the file).

Warning

Some people might be tempted to just use vi when opening logfiles; however, be warned that vi caches the entire file to disk when it opens it. This might not be a problem for small logfiles, but when you open a 2 GB Apache log, you might possibly fill up the remaining space on your disk! less does not cache the entire file like vi, so it is the safer choice for large logfiles.

grep is also a very useful tool for logs. grep accepts a pattern as an argument and will look for that pattern within a file. For instance:

$ grep dhclient /var/log/syslog
            

will return all of the log entries containing the word "dhclient". Grep is particularly useful for Xorg logs, as the warnings and errors in these logs are preceded by WW and EE, respectively. To grep out all of the warnings and errors from Xorg.0.log, type:

$ grep -E '(WW|EE)' /var/log/Xorg.0.log
            

Note that the -E option turned on extended regular expressions so that you could use a more advanced pattern.

Tip

Some logfiles you want to view might be gzipped due to log rotation. In this case, use zless and zgrep just as you would less and grep, respectively.

Mount Removable Devices with Persistent Names

Mount your USB drive so it appears as /media/music every time. Mount your FireWire drive so it appears as /media/data every time.

My storage setup on my Ubuntu box at home is a bit unusual. I have an external drive dedicated to music and another external drive that backs up the music drive. I also have an external drive to hold all my personal documents, pictures, and movies, with another external drive to back up that drive. Those four drives are all connected to my desktop via FireWire. Finally, I have an external drive that's a temp drive: when I download a new movie or set of pictures, or rip a new CD, I keep those files on the temp drive until I can properly place them on the music or personal data drive. Unlike the other four, the temp drive uses USB.

Five drives sounds cool, but there's a major annoyance associated with them. When I reboot Ubuntu (a rare occasion, to be sure, but it does happen), their mount points shift around. Sometimes FireWire drive number one gets /media/sdb1, and sometimes /media/sdc1. The same thing happens to the other drives as well, which wreaks havoc with my backup scripts and my attempts to SSH in from other machines, to name just two problems I've experienced. I want persistent naming of those drives, so that FireWire number one is always/media/<something>, FireWire number two is always /media/<somethingelse>, and so on.

After much searching, I found the answer on the Ubuntu forums, at http://ubuntuforums.org/showthread.php?t=91948, and in an excellent overview by Daniel Drake titled "Writing udev rules" (http://www.reactivated.net/writing_udev_rules.html). Basically, Ubuntu includes a technology called udev, which manages dynamic devices. I can tell udev how I want it to label each separate drive when I plug it in, a far more efficient and usable method than the default. Here's how to do so.

You're going to be editing a file found at /etc/udev/rules.d/10-local.rules. Check to see if that file already exists on your hard drive:

$ ls /etc/udev/rules.d/10-local.rules
         

If it's already there, then you're going to edit it; if it's not there currently, go ahead and create a blank file with the proper name and permissions:

$ sudo touch /etc/udev/rules.d/10-local.rules
$ sudo chmod 777 /etc/udev/rules.d/10-local.rules
         

At the end of the process described in this hack, the devices will be listed in the 10-local.rules file. The USB drive, for instance, will look like this:

BUS=="usb", SYSFS{serial}=="6R0000065086", NAME{all_partitions}=="temp"

The first part—BUS=="usb"—is obvious; that's how that drive connects to the machine. The third part—NAME{all_partitions}=="temp"—makes sense too, since "temp" is the name I want the drive to use in the future as its mount point, instead of /media/sdb1 or whatever else Ubuntu decides to use. But what about that second part? Where is "6R0000065086" coming from?

In order for udev to work, it has to know which specific drive I want to label "temp". When I plug in a drive (or any dynamic device), the drive identifies itself in a whole host of ways. I need to pick a unique data point that I can pass to udev to use for identification. To see what udev sees when you plug in a drive, I run the following command (notice that the path given is the current path that Ubuntu has assigned to the drive—in this case, /dev/sda):

$ udevinfo -a -p $(udevinfo -q path -n /dev/sda)
         

Once I press Enter, an enormous amount of data streams by, looking something like this (this listing has been greatly truncated for length):

device '/sys/block/sda' has major:minor 8:0
  looking at class device '/sys/block/sda':
    SUBSYSTEM=="block" 
    SYSFS{dev}=="8:0" 
    SYSFS{range}=="16" 
  SYSFS{removable}=="0" 
 SYSFS{size}=="488397168" 
    SYSFS{stat}==" 2159 102 16586 1871738 28 17 360 204 0 14257  1871942" 

...

  looking at the device chain at 
'/sys/devices/pci0000:00/0000:00:1e.0/0000:05:09.2/usb4/4-1/4-1.2/4-1.2:1.0':
 BUS=="usb" 
    ID=="4-1.2:1.0" 
    DRIVER=="usb-storage" 
    SYSFS{bAlternateSetting}==" 0" 
    SYSFS{bInterfaceClass}=="08" 
    SYSFS{bInterfaceNumber}=="00" 
    SYSFS{bInterfaceProtocol}=="50" 
    SYSFS{bInterfaceSubClass}=="06" 
    SYSFS{bNumEndpoints}=="02" 
    SYSFS{modalias}=="usb:v04B4p6830d0001dc00dsc00dp00ic08isc06ip50" 

  looking at the device chain at 
'/sys/devices/pci0000:00/0000:00:1e.0/0000:05:09.2/usb4/4-1/4-1.2':
    BUS=="usb" 
    ID=="4-1.2" 
    DRIVER=="usb" 
    SYSFS{bConfigurationValue}=="1" 
    SYSFS{bDeviceClass}=="00" 
    SYSFS{bDeviceProtocol}=="00" 
    SYSFS{bDeviceSubClass}=="00" 
    SYSFS{bMaxPower}=="  0mA" 
    SYSFS{bNumConfigurations}=="1" 
    SYSFS{bNumInterfaces}==" 1" 
    SYSFS{bcdDevice}=="0001" 
    SYSFS{bmAttributes}=="c0" 
    SYSFS{configuration}=="" 
    SYSFS{devnum}=="3" 
    SYSFS{idProduct}=="6830" 
    SYSFS{idVendor}=="04b4" 
    SYSFS{manufacturer}=="Cypress Semiconductor" 
    SYSFS{maxchild}=="0" 
    SYSFS{product}=="USB2.0 Storage Device" 
    SYSFS{serial}=="6R0000065086" 
    SYSFS{speed}=="480" 
    SYSFS{version}==" 2.00" 

...

See the line in bold that says SYSFS{serial}=="6R0000065086"? That's the unique serial number for this device, so that's exactly what I'll use in the line I'm adding to 10-local.rules. Notice also that the line BUS=="usb" is also present, in case I don't know how /dev/sda is connected to my box (hey, with five drives, it could happen!). Once again, then, the new line in /etc/udev/rules.d/10-local.rules is:

BUS=="usb", SYSFS{serial}=="6R0000065086", NAME{all_partitions}=="temp"

I need to repeat the process for any other drives as well. Although the next four drives are all connected with FireWire, the process is the same; it's just the listings that are different:

$ udevinfo -a -p $(udevinfo -q path -n /dev/sdb)

device '/sys/block/sdb' has major:minor 8:16
  looking at class device '/sys/block/sdb':
    SUBSYSTEM=="block" 
    SYSFS{dev}=="8:16" 
    SYSFS{range}=="16" 
 SYSFS{removable}=="0" 
 SYSFS{size}=="234441648" 
    SYSFS{stat}=="1166 102 8386 739882 27 16 344 52 0 6575 739934" 

...

  looking at the device chain at 
'/sys/devices/pci0000:00/0000:00:1e.0/0000:05:04.0/fw-host0/0030e0f4e020dca0/0030e0f4e020dca0-0':
 BUS=="ieee1394" 
    ID=="0030e0f4e020dca0-0" 
    DRIVER=="sbp2" 
  SYSFS{address}=="0x0000fffff0000830" 
    SYSFS{ignore_driver}=="0" 
    SYSFS{length}=="0" 
    SYSFS{model_id}=="0x000001" 
    SYSFS{model_name_kv}=="OXFORD IDE Device LUN 0 " 
    SYSFS{specifier_id}=="0x00609e" 
    SYSFS{version}=="0x010483" 

  looking at the device chain at 
'/sys/devices/pci0000:00/0000:00:1e.0/0000:05:04.0/fw-host0/0030e0f4e020dca0':
 BUS=="ieee1394" 
    ID=="0030e0f4e020dca0" 
    DRIVER=="unknown" 
 SYSFS{bus_options}=="IRMC_0_ CMC_0_ ISC_0_ BMC_0_ PMC_0_ GEN_0_ 
   LSPD_2_ MAX_REC_64_ MAX_ROM_0_ CYC_CLK_ACC_255_" 
    SYSFS{capabilities}=="0x0083c0" 
    SYSFS{guid_vendor_id}=="0x0030e0" 
 SYSFS{guid}=="0x0030e0f4e020dca0" 
    SYSFS{nodeid}=="0xffc0" 
    SYSFS{tlabels_allocations}=="1262" 
    SYSFS{tlabels_free}=="64" 
    SYSFS{tlabels_mask}=="0x05965272090596527209" 
    SYSFS{vendor_id}=="0x0030e0" 
    SYSFS{vendor_name_kv}=="Oxford Semiconductor Ltd.   " 

...

In this instance, udevinfo tells me how this drive is connected to my Ubuntu desktop: BUS=="ieee1394" (remember that IEEE1394 is the official name for FireWire). FireWire drives don't include SYSFS{serial} like USB drives do; instead, their unique identifiers are SYSFS{guid}. This particular drive holds my personal files, so I want it to show up as /media/data. Put all that together, and I add this line to /etc/udev/rules.d/10-local.rules:

BUS=="ieee1394", SYSFS{guid}=="0x0030e0f4e020e229", NAME{all_ ⏎
partitions}=="data"

After performing the same task three more times, /etc/udev/rules.d/10-local.rules looks like this:

BUS=="usb", SYSFS{serial}=="6R0000065086", NAME{all_partitions}=="temp" 
BUS=="ieee1394", SYSFS{guid}=="0x0030e0f4e020e229", 
NAME{all_partitions}=="data" 
BUS=="ieee1394", SYSFS{guid}=="0x0030e0f4e020dca0", 
NAME{all_partitions}=="data_copy" 
BUS=="ieee1394", SYSFS{guid}=="0x0030e0f4e020912c", 
NAME{all_partitions}=="music" 
BUS=="ieee1394", SYSFS{guid}=="0x0030e0f4e020c727", 
NAME{all_partitions}=="music_copy"

Now I need to test my changes to make sure they'll actually work. First, I need to restart udev, the subsystem that is managing all this craziness:

$ sudo /etc/init.d/udev restart
         

Now I'll test the rules for the USB external drive to make sure everything will work. To test, I use udevtest. The syntax of the command is:

$ sudo udevtest 
            
               sysfs_device_path
            
             
            
               subsystem
            
         

How do I know what the sysfs_device_path and subsystem are? Look back when I ran udevinfo -a -p $(udevinfo -q path -n /dev/sda) and notice the first things reported back to me:

looking at class device '/sys/block/sda':
  SUBSYSTEM=="block"

My udevtest command therefore looks like this:

$ sudo udevtest /sys/block/sda block
         

The results appear immediately below it (some error messages about other items in udev that aren't important here have been removed):

udevtest.c: looking at device '/block/sda' from subsystem 'block'
...
udevtest.c: opened class_dev->name='sda'
udev_rules.c: configured rule in '/etc/udev/rules.d/10-local.rules:1' applied, 'sda' becomes 'temp'
udev_add.c: creating device node '/dev/temp', major = '8', minor = '0', mode = '0640', uid = '0', gid = '46'
udev_add.c: creating device partition nodes '/dev/temp[1-15]'

Excellent! Now I know that udev would in fact create a device name /dev/temp—and therefore a mount point at /media/temp—instead of /dev/sda and /media/sda, and will always use that in the future. By the way, don't worry—even though the output says it was creating nodes, it really wasn't. It was just showing me what it would do when it runs for real.

So I know the USB drive works, but what about one of the FireWire drives?

$ sudo udevtest /sys/block/sdd block
         

Here are the results for that drive (again, some unnecessary data has been removed):

udevtest.c: looking at device '/block/sdd' from subsystem 'block'
...
udevtest.c: opened class_dev->name='sdd'
udev_rules.c: configured rule in '/etc/udev/rules.d/10-local.rules:5' applied, 'sdd' becomes 'music_copy'
udev_add.c: creating device node '/dev/music_copy', major = '8', minor = '48', mode = '0640', uid = '0', gid = '46'
udev_add.c: creating device partition nodes '/dev/music_copy[1-15]'

Exactly what I wanted, so it looks like everything will work as intended. Now I restart the machine, and lo and behold, my drives appear, just as I wanted them:

/media/temp
/media/data
/media/data_copy
/media/music
/media/music_copy

Of course, there's nothing stopping you from setting up udev so that your digital camera always shows up as /media/camera. Or so your iRiver music jukebox consistently mounts as /media/iriver. And so on. Basically, if your system sees it as a dynamic device, you can use udev to map it to a specific mount point. Thanks to udev, Ubuntu just got a lot easier to use.

Scott Granneman

Mount Remote Directories Securely and Easily

If you can access it via SSH, you can mount it on your filesystem, giving you secure access to your favorite stuff.

Samba has transformed how Penguinistas interact on networks with Windows machines and even other Linux boxes. In particular, I often use smbfs (short for "Samba filesystem") to mount a Samba share on my box for easy access. For example, I have two computers on my network: a server named sophocles and a laptop named euripides. A huge collection of music can be found on sophocles at /var/music, but I want easy access to those goodies from euripides. Using Samba on sophocles, I share /var/music, and using smbfs on euripides, I mount /var/music on my server sophocles to /home/scott/tunes on my laptop euripides. By doing so, it appears to me while I'm using euripides as though tunes is just another local folder on my laptop, so I can read files from it, save to it, and do anything else I could if that folder were in fact on my machine.

This is great, except that there are some issues. Setting up Samba can be a royal pain, so any time I can use something simpler, I jump at the chance. Second, Samba shares aren't secure by default. Call me paranoid, but I don't like anything flowing over a network that isn't encrypted. Yes, it's possible to tunnel Samba using SSH, but that just adds more time and trouble on top of the royal pain that Samba sometimes causes. Finally, Samba was designed for LANs, not the wild and woolly Internet, so accessing shares remotely is out of the question (yes, there are ways to do it, but it's just not a good idea on today's Net...and it causes yet more complication!).

But I'm here to tell you that there's a better way: sshfs, the SSH filesystem. Instead of Samba, it uses SSH. All the problems I listed previously are obviated when you switch to SSH. SSH is a breeze to set up and use. All SSH traffic is encrypted, hence the name Secure SHell. And SSH was designed for use on LANs as well as the big, bad Internet, so it's a great choice no matter where you are.

To go back to my original example, I can SSH from euripides to sophocles (and vice versa, for that matter), so I now use sshfs to mount /var/music on sophocles to /home/scott/tunes on euripides. It's easy, it's encrypted, and if this is a connection I'm going to need all the time, I can set things up in /etc/fstab to automate the whole process. Ready to try it out?

Before doing anything else, make sure that you can SSH from the client to the server:

$ ssh scott@sophocles
Linux sophocles 2.6.15-18-386 #1 PREEMPT Thu Mar 9 14:41:49 UTC 2006 i686 GNU/Linux

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
Last login: Sun Mar 12 13:59:45 2006
$ exit
         

If you can't SSH in, you need to set that up:

  • On an Ubuntu server, use sudo apt-get install ssh.
  • On Windows, install Cygwin, available from http://www.cygwin.com, and install OpenSSH as a service.
  • On the Mac, enable Remote Login in System Preferences→Sharing→Services.

If you can successfully SSH in to the server, you're finished with that machine (see how much simpler this is than it would be if you were using Samba?).

Now you need to focus on the local computer. Make sure you've enabled the universe archive in your /etc/apt/sources.list file, as detailed in "Modify the List of Package Repositories" [Hack #60]. Once you've done so, install sshfs:

$ sudo apt-get install sshfs
         

You'll be told that apt is going to have to install some additional packages in addition to sshfs (fuse-utils and libfuse2), so go ahead and approve apt's request.

When apt finishes installing your new software, it's time to fix some permissions so that normal users can mount and unmount using sshfs. If you leave out this step, mounting and unmounting will require the use of sudo and knowledge of the root password, which you probably don't want. Better to go ahead and run this command, which grants all users on your system execute permission for the fusermount command, used by sshfs to perform its magic:

$ sudo chmod +x /usr/bin/fusermount
         

Now you need to make the mount point you plan to use on your local machine. In my case, this would be /home/scott/tunes:

$ mkdir /home/scott/tunes
         

In order for sshfs to work, you have to load the fuse module. For now, go ahead and run this command:

$ sudo modprobe fuse
         

I know you don't want to have to run that command every time you want to mount with sshfs, so use the following command to tell Ubuntu to automatically load the fuse module when you boot:

$ sudo sh -c "echo 'fuse' >> /etc/modules"
         

Warning

Be very careful and use >> instead of > (in other words, append instead of overwrite)! If you accidentally use >, you will hose your box and none of your modules will load on boot. For that reason alone, you really should back up up /etc/modules first, with sudo cp /etc/modules /etc/modules_bak.

Now for the final step: mount /var/music on sophocles to /home/scott/tunes on euripides. The syntax of the command is as follows:

$ sshfs 
            
               user@
            
            [
            
               IP
            
            |
            
               HOSTNAME of remote machine
            
            ]
            
               :/shared/path /local/mount/point
            
         

If your username is the same on the remote and local machines, you can leave it off. DNS will work either on your LAN, if you have an entry for the server in your /etc/hosts file, or over the Net to a machine with a registered address. Otherwise, use the computer's IP address. If you don't specify a path after the colon, sshfs mounts the home directory for the SSH user, which may be all that you want. In my case, my username is the same on both boxes, I have an entry for sophocles in the /etc/hosts file, and I want to mount a specific directory on the computer, so my command looks like this:

$ sshfs sophocles:/var/music /home/scott/tunes
         

Warning

If you get the error "fusermount: failed to open /dev/fuse: Permission denied," add yourself to the fuse group with sudo addgroup username fuse, log out, log back in again, and try the command again.

Now I can open my favorite music player, point it to /home/scott/tunes, and start enjoying jazz, rock, country, or whatever it is that floats my boat. Remember, it's all secure, so I don't have to worry about anyone sniffing packets and finding out what's moving around on my network.

To unmount, you have a choice. You can use the fusermount command with the -u option (for "unmount"):

$ fusermount -u /home/scott/tunes
         

Or it might be simpler just to use the good ol' umount command:

$ sudo umount /home/scott/tunes
         

If I knew that I wanted to make my connection to sophocles whenever I booted, I could add the following to /etc/fstab:

sophocles:/var/music /home/scott/tunes fuse defaults 0 0

The first three items are different from the usual fstab entries, with the first two exactly like the sshfs command you just used, and fuse indicating to your computer the kind of filesystem it's going to mount. If you want to change the other entries and you know what you're doing (run man fstab to learn more), go right ahead.

Once you discover sshfs, you're going to use it all the time. There's no reason not to: it works beautifully, and the added benefit of security is icing on the cake. Thank you, sshfs!

Scott Granneman

Make Videos of Your Tech-Support Questions

Have a friend or relative with tech-support troubles? Show him how to record a movie of his Ubuntu problems and send it to you, you ever-helpful, all-knowing Ubuntu guru, you.

We've all been there: Uncle Gussy is on the phone to you trying to explain a computer problem he's having, but his description isn't helping in the slightest: "Welllll, I click on the thingie, and then this doohickey opens, so I click on that, and then the computer does something weird." Gee, thanks, Uncle Gussy!

If you were smart enough to set up Uncle Gussy with Ubuntu (and being a good niece or nephew, that's what you did, right?), you could just tell him to use Istanbul to record what he's doing on his computer into a patent-free, open source Ogg Theora video file. Uncle Gussy could then email you the file, which would vastly simplify your job as family tech support.

Tip

To find out more about Istanbul, or to keep up with its development, check out http://live.gnome.org/Istanbul. Don't know what Ogg Theora is, or are only familiar with Ogg Vorbis? Then go read Wikipedia's article on the subject, at http://en.wikipedia.org/wiki/Ogg_Theora.

When you first set up Uncle Gussy's Ubuntu box, make sure Istanbul is installed. First look for it, to see if it's already on the machine:

$ whereis istanbul
istanbul:

Nope. OK, you need to put it on there. Make sure you have the universe repository enabled [Hack #60], and then install Istanbul:

$ sudo apt-get install istanbul
         

You may find that apt wants to install several other packages as well, chiefly related to gstreamer, so go ahead and indicate your approval. Once apt has finished its business, you need to start Istanbul so you can configure it, so select Applications→Sound & Video→Istanbul Desktop Session Recorder. You'll know Istanbul has loaded when you see a large red dot to the left of the clock on the top panel, as shown in Figure 8-15.

Figure 8-15. Istanbul on the top-edge panel

Istanbul on the top-edge panel

Right-click on Istanbul's icon and choose Preferences, which brings up the window shown in Figure 8-16.

Figure 8-16. Istanbul Preferences

Istanbul Preferences

It's a good idea to check the box next to "Encode after stopping record (uses less CPU)" for the reason given in the setting's name. Also, to reduce file size, change Frames Per Second from 10.0 to 1.0. Of course, this also means that Uncle Gussy will have to move slowly and deliberately as he records his movie, but that can only help you when you're viewing it. As for the Network Stream section, that would more than likely be far too complicated for the likes of Uncle Gussy, so just leave those prefs alone. Click Close, head on home, and then wait patiently until the next phone call from Uncle Gussy.

Tip

You should test Istanbul before Uncle Gussy uses it, to make sure it will work correctly. If Istanbul doesn't quite work, try playing with the Preferences. In particular, uncheck "Encode after stopping record," which sometimes fixes otherwise inexplicable Istanbul problems. This will take up more CPU, as Istanbul will encode the file on the fly if you uncheck this box.

The Ogg Theora encoder loves to gobble up CPU cycles, so whatever you can do to reduce that load would be good. Checking the box next to "Encode after stopping record" was a good start, but you should also walk Uncle Gussy through temporarily changing his screen resolution to 800x600 if at all possible.

Now for Uncle Gussy's debut as Cecil B. DeMille. Tell him to start Istanbul by going to Applications→Sound & Video→Istanbul Desktop Session Recorder. To start recording, he clicks on the large red circle, which changes to a stop box to indicate that recording is in progress. He then performs the actions he wants you to watch and decipher. Tell him to go slow and make all his actions deliberate, so that you can easily comprehend what he's doing.

When he's finished, he clicks on the stop box, and the icon changes once again, to a gray icon that GNOME uses to represent a hard drive, to let him know that Istanbul is encoding. If your uncle has a fast computer, or his movie is short, he may not even see that last icon, in which case he'll be back to the red circle. If Uncle Gussy is done with Istanbul, he can close the program by right-clicking on the icon and choosing Quit.

There should now be a new file named desktop-recording.ogg in Uncle Gussy's home directory. Tell him to email that to you, and shortly thereafter you'll be able to see exactly what's going on with his Ubuntu machine. It's the next best thing to sitting next to him while he screws up—I mean, uses—his computer.

You may find that Istanbul is still a bit buggy, but it's getting a lot of attention from developers, so keep an eye on it. More and more, developers of various Linux distros are going to rely on Istanbul when users complain about various bugs and problems.

Tip

Istanbul is supposed to work with KDE, and therefore Kubuntu, but it's impossible at this time to access the Preferences menu by right-clicking on the Istanbul icon. Other than that, the program works, so make any changes to Preferences while in GNOME and then switch over to KDE to actually use the program.

Scott Granneman

Synchronize Files Across Machines

If you switch between multiple machines, it's a real hassle to keep track of which system has the latest version of a file. Use the free Unison File Synchronizer to keep all your systems in sync, even Windows and Mac machines.

There's an old saying that everyone should take to heart: "There are two kinds of computer users: those who have lost data and those who are about to." I belong in that first camp, but it only took one nasty experience with an accidentally deleted term paper to force me to resolve never to let that happen again. That night long ago, when I lost the fruits of my labors analyzing the great film Raising Arizona, taught me the hard way that backing up is essential. And while traditional backups [Hack #79] are vital, keeping two or more machines in sync not only gives you some extra security, but it also makes it easy to switch from system to system without having to remember which one has the latest copy of a file.

If a user new to Linux starts asking more experienced users about a good way to sync his data, he will soon hear about a wonderful tool named rsync. rsync was developed by Andrew "Tridge" Tridgell, the same man behind Samba; in fact, Tridge has stated that he believes he'll be remembered through posterity for rsync far more than for Samba, and he just may be right. However, rsync, while absolutely excellent, is not necessarily the best software to use for syncing two repositories that you're actively working with.

The Problem with rsync

rsync is truly awesome software, and it really is worth your time to check it out. However, I don't use rsync for my day-to-day sync needs. Instead, I use Unison, which uses rsync as its base. Why Unison instead of rsync? Because Unison synchronizes in two directions at one time, while rsync goes only in one direction.

Let me give another example, from my own setup. At home I have a desktop machine, but I also use a laptop. I work a lot outside of my home, so my laptop travels with me constantly; when I'm home, though, I sometimes leave my laptop in my backpack and use my desktop instead. I have a large amount of data that I need to have available to me at all times: web pages I've read, instruction manuals, articles I'm working on, photos, and so on. All told, it's about 10 GB worth of stuff. I keep all of this work on my desktop, and I keep a mirror of the same thing on my laptop.

Obviously, I need to keep all 10 GB in sync between my two machines. If I delete a file on the desktop, I need to delete it on the laptop. If I change a file on the laptop, it wouldn't do to open the original, unchanged file on the desktop and make changes there. That way lies madness.

So why can't I use rsync? Because my goal is to synchronize between two machines, each of which acts as a source and a destination at the same time. Here's one example: on my laptop, I move the file widgets from the directory foo to the directory bar. On the desktop, however, widgets is still in foo. If I run rsync with the laptop as the source and the desktop as the destination, so far so good: widgets is now in bar on both the laptop and the desktop. However, widgets is still in foo on the desktop. The next time I run rsync with the desktop as the source and the laptop as the destination, widgets gets copied over from foo on the desktop back into foo on the laptop, leaving me with widgets in both foo and bar, which produces a real mess.

Of course, I could run rsync with the --delete option set, so that files deleted on the source are also deleted on the destination, but that can be a very dangerous practice. What if I delete foo on the laptop but change foo on the desktop? If I run rsync with the laptop as source, then foo is deleted on the desktop, which is not what I want. Instead, I have to remember to run rsync with the desktop as source so that the changed foo is copied over to the laptop. But what if there were files I changed on the laptop and deleted on the desktop? Then I need to run rsync with the laptop as source...and on and on, ad infinitum, back and forth.

Unison solves this problem. You define a source and a destination, and then you run Unison. After some length of time, Unison starts asking you questions about the files it has found: copy this file from laptop to desktop? Copy this other file from desktop to laptop? Delete this file on the laptop since it was deleted on the desktop? You can accept Unison's guesses, specify a direction for the copy to go, or even tell Unison to skip the file entirely until another day. With Unison, synchronizing two directories, each on a different machine, becomes a far simpler task. And, as icing on the cake, Unison is cross-platform, as it runs on Linux, Mac OS X (with a few caveats; see "Further Information About Unison" at the end of this hack), Unix, and Windows machines.

Synchronize Files on Two Machines with Unison Using SSH

Installing Unison is easy as pie (however, make sure you have the universe repository enabled [Hack #60]). Just run the following command, and you're done:

$ sudo apt-get install unison unison-gtk
            

Now you have both the basic, command-line-only Unison program and the GTK-based GUI.

It's time to sync a group of files with another copy of those same files on another machine. For added safety, I'm going to use SSH for the transport, a capability built into Unison. Unfortunately, Unison doesn't create entries in the GNOME Applications menu or the KDE K menu, so you'll need to add those yourself. In the meantime, you can type unison-gtk on the command line. When the program opens, you'll see Figure 8-17, which asks you to either select or create a profile.

Figure 8-17. Select or create a Unison profile

Select or create a Unison profile

Click "Create new profile," and you're asked to give a name for the new profile. Since I'm going to sync up my folder that contains the work I've done on this book, I'm going to cleverly name this profile ubuntu_hacks and click OK. Now both profiles, default and ubuntu_hacks, appear. I double-click on ubuntu_hacks, and a small window opens, shown in Figure 8-18, asking me to enter the local directory that I want to sync.

Figure 8-18. Select the local directory you want to sync

Select the local directory you want to sync

Since I don't like to type when I can avoid it, I instead click Browse and navigate to the folder:

/home/rsgranne/documents/clientele/aaa_current/OReilly/ubuntu_hacks

I click OK, and another window opens, shown in Figure 8-19. This one wants to know about the remote folder.

Figure 8-19. Select the remote directory you want to sync

Select the remote directory you want to sync

Notice that I can sync to a local folder (which could also be a folder accessed through Samba or NFS that I've mounted on this machine), or to another machine via SSH, RSH, or a socket. In this case, I'm using SSH, so I enter the address of the remote host—in this case, 192.168.0.14—and the username, which is rsgranne. Finally, I type in the remote directory I want to sync:

/media/data/zzz_rsg/documents/clientele/aaa_current/OReilly/ubuntu_hacks

I click OK, and since this is the first time I've synched these two folders, Unison presents me with a warning that it is going to have to create archives for those two folders from scratch. No problem, so I click OK. The main Unison window opens, and Unison goes to work, comparing the two directories (and all their subdirectories as well, since the program automatically recurses). Be patient during this process. Unison may appear to be locked up, but it's just taking its time to digest all the files and folders. The more data, the longer Unison will take. Eventually, Figure 8-20 appears.

Figure 8-20. You can sync files in a number of directions

You can sync files in a number of directions

Unison's interface is pretty self-explanatory, with the arrows showing the direction in which copying will occur. If I want to switch the direction in which a file is copied, I simply select it and then click the "Right to Left" or "Left to Right" button. If I want Unison to ignore a file, I click Skip. If I want to invoke a diff-type program to actually compare the contents of the two files (this obviously works only with text files and the like) and then selectively merge the differences, I click Diff and then Merge. When I'm sure that everything is the way I want it, I click the Go button to tell Unison to copy files and folders. There are actually lots of other options available through the Actions menu, so it's a good idea to check it out.

Tweaking Your Profiles

When I first started Unison, I had to create a profile, which keeps track of the directories I want to compare and the transport method, among other configurations. Profiles live in a hidden directory (.unison), which itself is in your home directory, and they're just text files that end with the .prf extension. Before I finish looking at Unison, I want to mention a few other tweaks that you should add manually to those files in order to make your sync processes work a little more smoothly.

In order to add these tweaks, you're going to need to open your profiles with your favorite text editor. In my case, I'm going to open and change ubuntu_hacks.prf, located in /home/rsgranne/.unison.

To start, I'll add the following to the file (it doesn't matter where you put these preferences, but I like to put them at the top of the file, after my two roots and before any "ignore" lines):

times = true

If you use Unison for any length of time, you're going to notice that a lot of files need their "props" synchronized. Complaining about "props" is just Unison's way of telling you that the date- and timestamps for your files—which indicate when the files were modified—are not exact. By setting times to true, you are copying over not just the contents of the files, but their modification dates and times as well, which will vastly reduce any "props" complaints that you'll see.

Tip

This setting copies over modification timestamps for files only, not directories.

In a similar vein, you may want to add the following:

owner = true
group = true

When you add these two settings to your profile, you order Unison to synchronize the owners and groups of files, along with their contents, which can really help to keep things straight as you move files around between machines.

If you want to synchronize users and groups by the real numbers that your Ubuntu system uses instead of the names that we humans use (in other words, by 1000 instead of rsgranne; if you don't know what I'm talking about, take a look at your /etc/passwd file and run the command man 5 passwd), then add this preference to your profile as well:

numericids = true

If you're using Unison to sync only Linux boxes, then you can skip this next preference. But if you're involving Windows machines in your sync—say, from a Windows workstation to a Linux machine, or from a Windows PC to a Linux workstation—then you probably want to use fastcheck (but read the following paragraphs carefully):

fastcheck = yes

You'll probably want to use fastcheck = yes most of the time, but every once in a while you'll want to comment out that line with a pound sign in front of it (#), run Unison, and then remove the pound sign. Let me explain why.

You may already have asked yourself a pretty important question: how the heck does Unison know that a file has changed? For files on a Linux box, Unison looks at their inode numbers and modtimes; if either of those is different, then Unison thinks that the file has changed. Windows boxes, of course, don't have inode numbers, so Unison's default for Windows is to scan the contents of each file to see if anything has changed. This is certainly a safe method, but if you have a lot of files to scan on a Windows machine, it's going to take a long, long time. Failing to set the fastcheck preference then, or setting it to fastcheck = default, results in this behavior, which probably is not what you want to do.

Tip

Don't know what an inode number is? Here's a handy explanation: http://en.wikipedia.org/wiki/Inode.

If you set fastcheck to yes, however, then Unison acts differently on your Windows machines: it looks at the file modification times instead of scanning the contents of each file (Linux files still get the same treatment: their inode numbers and modtimes are checked). This results in a much faster scan of your Windows machine, but in very rare circumstances (a file has been changed, but you have somehow managed not to change the create time, the modification time, and the length of the file—pretty hard to do), Unison may refuse to make an update of some of the files on the Windows machine.

Now, practically speaking, you don't have to worry about this very much. First, you'd have to really jump through hoops to fool Unison, and second, even if Unison thought that it should possibly overwrite such a file, it won't, as the Unison manual explains: "Unison will never overwrite such an update with a change from the other replica, since it always does a safe check for updates just before propagating a change."

Since this is the case, I would recommend leaving fastcheck set to yes most of the time, but, every once in a while, comment out that preference with a pound sign, so that the full contents of your Windows files will be checked on the next run. Then, after you've satisfied yourself that things are copacetic, uncomment fastcheck = yes and go back to the faster method.

Changing the Location of Your Logfile Directory

When you run Unison, the program generates a logfile, which can be a very useful thing. The default, however, is to create this logfile—named unison.log—in your home directory, which I find annoying since I try to keep my home directory neat and tidy. If you don't mind this, then leave the logfile preference alone. But if you, like me, want to change the location in which the logfile goes, then set this preference:

logfile = $HOME/.unison/unison.log

I like the logfile to go into the hidden .unison directory, but you can change this any way you like.

Further Information About Unison

The main site for all things Unison can be found at http://www.cis.upenn.edu/~bcpierce/unison/. Hopefully, it'll get a cooler URL someday.

On the main Unison site, you can read the very complete and very informative Unison documentation (if you really get into Unison, then you should look into running it sans GUI; the documentation will tell you everything you need to know, although the GUI actually provides a good way to learn the basics). If that doesn't help, there is a good FAQ that covers several important questions, especially one that I was wondering myself: "Does Unison work on Mac OS X?" Be sure to read that answer in full before trying to use Unison on that OS!

For further support, there are a few listservs you can join—or just search, if you don't want to join—at Yahoo! Groups. Unison-users is just that: a list for users of Unison. It currently has over 600 members, and they produce in the neighborhood of 100 messages a month, so it's not too overwhelming. You can get more information about the list at http://groups.yahoo.com/group/unison-users/. If you're a developer, then you might want to check out Unison-hackers at http://lists.seas.upenn.edu/mailman/listinfo/unison-hackers. If all you want are the announcements when a new version is released, then you should subscribe to the low-volume (one message per month) Unison-announce, which you can find at http://groups.yahoo.com/group/unison-announce—or just send a blank email to unison-announce-subscribe@groups.yahoo.com.

Finally, Open magazine, a "weekly e-zine for Linux and Open Source computing in the enterprise," has a nice article about Unison available at http://www.open-mag.com/features/Vol_53/synch/synch.htm. It focuses on getting Unison to work with Windows as well as Linux, and includes some technical details about fail-safe provisions that Unison provides.

Scott Granneman

Personal tools