Advanced Filesystem Implementor's Guide : Introducing ext3

Advanced Filesystem Implementor's Guide : Introducing ext3 Daniel Robbins David H. Askew With the 2.4 release of Linux come a host of new filesystem possibilities, including Reiserfs, XFS, GFS, and others. These filesystems sound cool, but what exactly can they do, what are they good at, and exactly how do you go about safely using them in a production Linux environment? Daniel Robbins answers these questions by showing you how to set up these new advanced filesystems under Linux 2.4. In this installment, Daniel takes a look at ext3, a new improved version of ext2 with journaling capabilities. 1.0 2005-08-13 Introduction

The original version of this article was first published on IBM developerWorks, and is property of Westtech Information Services. This document is an updated version of the original article, and contains various improvements made by the Gentoo Linux Documentation team.

In the past few installments, we've taken a bit of a detour by looking at non-traditional filesystems such as tmpfs and devfs. Now, it's time to get back to disk-based filesystems, and we do this by taking a look at ext3. The ext3 filesystem, designed by Dr. Stephen Tweedie, is built on the framework of the existing ext2 filesystem; in fact, ext3 is very similar to ext2 except for one small (but important) difference -- it supports journaling. Yet even with this small addition, I think you'll find that that ext3 has several surprising and intriguing capabilities. In this article, I'll give you a good understanding of how ext3 compares to the other journaling filesystems currently available. In my next article, we'll get ext3 up and running.

Understanding Ext3

So, how does ext3 compare to ReiserFS? In previous articles, I explained how ReiserFS is well suited to handling small files (under 4K), and in certain situations, ReiserFS' small file performance is ten to fifteen times greater than that of ext2 and ext3. However, while ReiserFS has many strengths, it also has weaknesses. In the current implementation of ReiserFS (version 3.6), certain file access patterns can actually result in significantly worse performance than ext2 and ext3, particularly when reading large mail directories. Also, ReiserFS doesn't have a good track record of NFS compatibility and has poor sparse file performance. In contrast, ext3 is a very well-rounded filesystem. It's a lot like ext2; it's not going to give you the blazingly fast small-file performance that ReiserFS gives you, but it's not going to give you any unexpected performance or functionality hiccups either.

One of the nice things about ext3 is that because it is based on the ext2 code, ext2 and ext3's on-disk format is identical; this means that a cleanly unmounted ext3 filesystem can be remounted as an ext2 filesystem with absolutely no problems. And that's not all. Thanks to the fact that ext2 and ext3 use identical metadata, it's possible to perform in-place ext2 to ext3 filesystem upgrades. Yes, you read that right. By upgrading a few key system utilities, installing a modern 2.4 kernel and typing in a single tune2fs command per filesystem, you can convert your existing ext2 servers into journaling ext3 systems. You can even do this while your ext2 filesystems are mounted. The transition is safe, reversible, and incredibly easy, and unlike a conversion to XFS, JFS, or ReiserFS, you don't need to back up and recreate your filesystems from scratch. Now, for a moment, consider the thousands of production ext2 servers in existence that are just minutes away from an ext3 upgrade; then, you'll have a good grasp of ext3's importance to the Linux community.

If I had to describe ext3 in one word, I'd call it "comfortable". It's incredibly easy to ext3-enable an existing ext2 system, and after you do, you're not going to run into any unexpected performance quirks. And there's yet another way that ext3 excels in the comfort department; ext3 happens to be one of the most reliable journaled filesystems available for Linux, as I explain below.

Ext3 reliability

In addition to being ext2-compatible, ext3 inherits other benefits by sharing ext2's metadata format. For one, ext3 users gain access to a rock-solid fsck tool. You'll recall that one of the points of using a journaling filesystem is to avoid the need for an exhaustive fsck in the first place; however if you do end up getting corrupt metadata, either from a flaky kernel, bad hard drive, or something else, you'll greatly appreciate the fact that ext3 inherits ext2's fsck. In contrast, ReiserFS' fsck is in its infancy, and fixing flaky metadata when it does show up can be a difficult and dangerous process.

Metadata-only journaling

Interestingly, ext3 handles journaling very differently than ReiserFS and other journaling filesystems do. With ReiserFS, XFS, and JFS, the filesystem driver journals metadata, but makes no provisions for journaling data. With metadata-only journaling, your filesystem metadata is going to be rock solid, and you will probably never need to perform an exhaustive fsck. However, unexpected reboots and system lock-ups can result in significant corruption of recently-modified data. Ext3 uses a couple of innovative solutions to avoid these problems, which we'll look at in a bit.

But first, it's important to understand exactly how metadata-only journaling could end up biting you. As an example, let's say that you were modifying a file called /tmp/myfile.txt when the machine unexpectedly locked up, forcing a reboot. If you were using a metadata-only journaling filesystem such as ReiserFS, XFS or JFS, your filesystem metadata would be easily repaired, thanks to the metadata journal, and you wouldn't need to sit through a laborious fsck.

However, there's the distinct possibility that when you load /tmp/myfile.txt into a text editor, your file will not simply be missing recent changes, but will contain a good amount of garbage and may even be completely unreadable. This isn't something that will always happen, but it could happen and often does.

Here's why. Typical journaled filesystems like ReiserFS, XFS, and JFS take extra special care of metadata, but don't pay too much attention to data. In our above example, the filesystem driver was in the process of modifying several filesystem blocks. The filesystem driver updated the appropriate metadata, but didn't have time to flush the data from its caches to the new blocks on disk. Thus, when you loaded up /tmp/myfile.txt into a text editor, part or all of the file contained garbage -- blocks of data that didn't get initialized in time before the system locked up.

The ext3 approach

Now that we have a good general understanding of this problem, let's look how ext3 implements journaling. In ext3, the journaling code uses a special API called the Journaling Block Device layer, or JBD. The JBD has been designed for the express purpose of implementing a journal on any kind of block device. Ext3 implements its journaling by "hooking in" to the JBD API. For example, the ext3 filesystem code will inform the JBD of modifications it is performing, and will also request permission from the JBD before modifying certain data on disk. By doing so, the JBD is given the appropriate opportunities to manage the journal on behalf of the ext3 filesystem driver. It's quite a nice arrangement, and because the JBD is being developed as a separate, generic entity, it could be used to add journaling capabilities to other filesystems in the future.

1.1 xml/htdocs/doc/en/articles/prompt-magic.xml file : http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/doc/en/articles/prompt-magic.xml?rev=1.1&content-type=text/x-cvsweb-markup&cvsroot=gentoo plain: http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/doc/en/articles/prompt-magic.xml?rev=1.1&content-type=text/plain&cvsroot=gentoo Index: prompt-magic.xml =================================================================== Prompt magic Daniel Robbins Joshua Saddler Why stick with the standard boring shell prompt when you can easily make it colorful and more informative? In this tip, Daniel Robbins will show you how to get your shell prompt just the way you like it, as well as how to dynamically update your X terminal's title bar. 1.0 2005-08-21 Enhancing the system prompt

As Linux/UNIX people, we spend a lot of time working in the shell, and in many cases, this is what we have staring back at us:

bash-2.04$

If you happen to be root, you're entitled to the "prestige" version of this beautiful prompt:

bash-2.04#

These prompts are not exactly pretty. It's no wonder that several Linux distributions have upgraded their default prompts that add color and additional information to boot. However, even if you happen to have a modern distribution that comes with a nice, colorful prompt, it may not be perfect. Maybe you'd like to add or change some colors, or add (or remove) information from the prompt itself. It isn't hard to design your own colorized, tricked-out prompt from scratch.

Prompt basics

Under bash, you can set your prompt by changing the value of the PS1 environment variable, as follows:

$ export PS1="> "
>

Changes take effect immediately, and can be made permanent by placing the export definition in your ~/.bashrc file. PS1 can contain any amount of plain text that you'd like:

$ export PS1="This is my super prompt > "
This is my super prompt >

While this is, um, interesting, it's not exactly useful to have a prompt that contains lots of static text. Most custom prompts contain information like the current username, working directory, or hostname. These tidbits of information can help you to navigate in your shell universe. For example, the following prompt will display your username and hostname:

$ export PS1="\u@\H > "
drobbins@freebox >

This prompt is especially handy for people who log in to various machines under various differently-named accounts, since it acts as a reminder of what machine you're actually on and what privileges you currently have.

In the above example, we told bash to insert the username and hostname into the prompt by using special backslash-escaped character sequences that bash replaces with specific values when they appear in the PS1 variable. We used the sequences \u (for username) and \H (for the first part of the hostname). Here's a complete list of all special sequences that bash recognizes (you can find this list in the bash man page, in the "PROMPTING" section):

\aThe ASCII bell character (you can also type \007)\dDate in "Wed Sep 06" format\eASCII escape character (you can also type \033)\hFirst part of hostname (such as "mybox")\HFull hostname (such as "mybox.mydomain.com")\j The number of processes you've suspended in this shell by hitting ^Z \lThe name of the shell's terminal device (such as "ttyp4")\nNewline\rCarriage return\sThe name of the shell executable (such as "bash")\tTime in 24-hour format (such as "23:01:01")\TTime in 12-hour format (such as "11:01:01")\@Time in 12-hour format with am/pm\uYour username\vVersion of bash (such as 2.04)\VBash version, including patchlevel\w -- gentoo-doc-cvs@gentoo.org mailing list

Sequence	Description