Operating Systems

Start Lecture #24

The Unix Inode-based Filesystem

Continuing the idea of adapting storage schemes from other regimes to file storage, why don't we mimic the idea of (non-demand) paging and have a table giving, for each block of the file, where on the disk that file block is stored? In other words a ``file block table'' mapping each file block to its corresponding disk block. This is the idea of (the first part of) the unix i-node solution, which we study next.

Although Linux and other Unix and Unix-like operating systems have a variety of file systems, the most widely used Unix file systems are i-node based as was the original Unix file system from Bell Labs.

There is some question of what the i stands for. The consensus seems to be index. Now, however, people often write inode (not i-node) and don't view the i as standing for anything. Inode based systems have the following properties.

1. The directory entry for a file contains a pointer to the file's i-node.
   
   
2. The metadata for a file or directory is stored in the corresponding inode.
   
   
3. The inode itself points to the first few data blocks, often called direct blocks. (As of June 2006, the main Linux file system organization uses twelve direct blocks.) In the diagram on the right, the inodes contain pointers to six direct blocks and all data blocks are colored blue.
   
   
4. The inode also points to an indirect block, which then points to disk blocks. In the diagram, the indirect blocks (also called single indirect blocks) are colored green.
   
   
5. The inode also points to a double indirect block, which points an indirect block, which points to data blocks. In the diagram, double indirect blocks are colored magenta.
   
   
6. For some implementations there is a triple indirect block as well. A triple indirect block points to double indirect blocks, which point ... . In the diagram, the triple indirect block is colored yellow.
   
   
7. The i-node is in memory for open files. So references to direct blocks require just one I/O.
   
   
8. For big files most references require two I/Os (indirect + data).
   
   
9. For huge files most references require three I/Os (double indirect, indirect, and data).
   
   
10. For humongous files most references require four I/Os.
    
    
11. Actually, fewer I/Os are normally required due to caching.

Retrieving a Block in an Inode-Based File System

Given a block number (byte number / block size), how do you find the block?

Specifically assume

1. The file system does not have a triple indirect block.
2. We desire block number N, where N=0 is the first block.
3. There are D direct pointers in the inode. These pointers are numbered 0..(D-1).
4. There are K pointers in each indirect block. These pointers are numbered 0..(K-1).

    If N < D           // This is a direct block in the i-node
       use direct pointer N in the i-node
    else if N < D + K  // The single indirect block has a pointer to this block
       use pointer D in the inode to get the indirect block
       the use pointer N-D in the indirect block to get block N
    else   // This is one of the K*K blocks obtained via the double indirect block
       use pointer D+1 in the inode to get the double indirect block
       let P = (N-(D+K)) DIV K      // Which single indirect block to use
       use pointer P to get the indirect block B
       let Q = (N-(D+K)) MOD K      // Which pointer in B to use
       use pointer Q in B to get block N
  

For example, let D=12, assume all blocks are 1000B, assume all pointers are 4B. Retrieve the block containing byte 1,000,000.

• K = 1000/4 = 250.
• Byte 1,000,000 is in block number N=1000.
• N > D + K so we need the double indirect block.
• Follow pointer number D+1=13 in the inode to retrieve the double indirect block.
• P=(1000-262) DIV 250 = 738 DIV 250 = 2.
• Follow pointer number P=2 in the double indirect block to retrieve the needed single indirect block.
• Q=(1000-262) MOD 250 = 738 MOD 250 = 238.
• Follow pointer number 238 in the single indirect block to retrieve the desired block (block number 1000).

With a triple indirect block, the ideas are the same, but there is more work.

Homework: Consider an inode-based system with the same parameters as just above, D=12, K=250, etc.

1. What is the largest file that can be stored.
2. How much space is used to store this largest possible file (assume the attributes require 64B)?
3. What percentage of the space used actually holds file data?
4. Repeat all the above, now assuming the file system supports a triple indirect block.

End of Problem

4.3.3 Implementing Directories

Recall that the primary function of a directory is to map the file name (in ASCII, Unicode, or some other text-based encoding) to whatever is needed to retrieve the data of the file itself.

There are several ways to do this depending on how files are stored.

• For sequentially allocated files, the directory entry for a file contains the starting address on the disk and the file size. Since disks can only be accessed by sectors, we store the sector number. The system can choose to start all files on a block (rather than sector) boundary in which case the block number, which is smaller, is stored instead.
• For linked allocation (pure linked or FAT-based) the directory entry again points to the first block of the file.
• For inode-based file systems, the directory entry points to the inode.

Another important function is to enable the retrieval of the various attributes (e.g., length, owner, size, permissions, etc.) associated with a given file.

• One possibility is to store the attributes in the directory entry for the file. Windows does this.
• Another possibility for inode-based systems, is to store the attributes in the inode as we have suggested above. The inode-based file systems for Unix-like operating systems do this.

Homework: 25

Long File Names

It is convenient to view the directory as an array of entries, one per file. This view tacitly assumes that all entries are the same size and, in early operating systems, they were. Most of the contents of a directory are inherently of a fixed size. The primary exception is the file name.

Early systems placed a severe limit on the maximum length of a file name and allocated this much space for all names. DOS used an 8+3 naming scheme (8 characters before the dot and 3 after). Unix version 7 limited names to 14 characters.

Later systems raised the limit considerably (255, 1023, etc) and thus allocating the maximum amount for each entry was inefficient and other schemes were used. Since we are storing variable size quantities, a number of the consideration that we saw for non-paged memory management arise here as well.

Searching Directories for a File

The simple scheme is to search the list of directory entries linearly, when looking for an entry with a specific file name. This scheme becomes inefficient for very large directories containing hundreds or thousands of files. In this situation a more sophisticated technique (such as hashing or B-trees) is used.

4.3.4 Shared (Multinamed) Files (Links)

We often think of the files and directories in a file system as forming a tree (or forest). However in most modern systems this is not necessarily the case, the same file can appear in two different directories (not two copies of the file, but the same file). It can also appear multiple times in the same directory, having different names each time.

I like to say that the same file has two different names. One can also think of the file as being shared by the two directories (but those words don't work so well for a file with two names in the same directory).

• Shared files is Tanenbaum's terminology.
• If a file exists, one can create another name for it (quite possibly in another directory).
• This is often called creating another link to the file.
• Unix has two flavor of links, hard links and symbolic links (a.k.a. symlinks).
• Dos/windows has shortcuts, which behave somewhat like symlinks, but I don't believe it has an analogue of hard links.
• We will concentrate on both flavors of unix links.
• These links often cause confusion, but I really believe that the diagrams I created make it all clear.

Hard Links

With unix hard links there are multiple names for the same file and each name has equal status. The directory entries for both names point to the same inode.

• Hard links are thus symmetric multinamed files.
• When a hard link is created another name is created for the same file. The number of files in the system is the same before and after the hard link is created.
• It is not, I repeat NOT, true that one name is the real name and the other one is just a link.
• Indeed after the hard link has been created it is not possible to tell which was the original name and which is the newly created link.

For example, the diagram on the right illustrates the result that occurs when, starting with an empty file system (i.e., just the root directory) one executes

    cd /
    mkdir /A; mkdir /B
    touch /A/X; touch /B/Y
  

The diagrams in this section use the following conventions

• Yellow circles represent ordinary files.
• Blue squares represent directories.
• Names are written on the edges. For example, one name for the left circle is /A/X.
  • It is not customary to write the names on the edges, normally they are written in the circles and squares.
  • When there are no multi-named files, it doesn't matter if they are written in the node or on the edge.
  • We will see that when files can have multiple names it is much better to write the name on the edge.

Now we execute

    ln /B/Y /A/New
  

At this point there are two equally valid name for the right hand yellow file, /B/Y and /A/New. The fact that /B/Y was created first is NOT detectable.

• The directory entries for both file names point to the same i-node.
• The file has only one owner (the one who created the file initially).
• The file has one date of last access (the last access by any of its names), one set of permissions, one ... .
• Note the usage: the file nameS (plural) vs the file (singular).

Assume Bob created /B and /B/Y and Alice created /A, /A/X, and /A/New. Later Bob tires of /B/Y and removes it by executing

    rm /B/Y
  

The file /A/New is still fine (see third diagram on the right). But it is owned by Bob, who can't find it! If the system enforces quotas Bob will likely be charged (as the owner), but he can neither find nor delete the file (since Bob cannot unlink, i.e. remove, files from /A).

If, prior to removing /B/Y, Bob had examined its link count (an attribute of the file), he would have noticed that there is another (hard) link to the file, but would not have been able to determine in which directory (/A in this case) the hard link was located or what is the name of the file in that directory (New in this case).

Since hard links are only permitted to files (not directories) the resulting file system is a dag (directed acyclic graph). That is, there are no directed cycles. We will now proceed to give away this useful property by studying symlinks, which can point to directories.

Symlinks

As just noted, hard links do NOT create a new file, just another name for an existing file. Once the hard link is created the two names have equal status.

Symlinks, on the other hand DO create another file, a non-regular file, that itself serves as another name for the original file. Specifically

• Creation of a symlink results in an asymmetric multi-named file. Assuming the original was a regular file, the symlink does indeed have a different status.
• When a symlink is created another file is created. The contents of the new file is the name of the original file.
• A hard link in contrast is (another name for) the original file.
• The examples will make this clear.

Again start with an empty file system and this time execute the following code sequence (the only difference from the above is the addition of a -s).

    cd /
    mkdir /A; mkdir /B
    touch /A/X; touch /B/Y
    ln -s /B/Y /A/New
  

We now have an additional file /A/New, which is a symlink to /B/Y.

• The file named /A/New has the name /B/Y as its data (not metadata).
• The system notices that /A/New is a red diamond (symlink) so reading /A/New will return the contents of /B/Y (assuming the reader has read permission for /B/Y). It is also possible to read the contents of /A/New itself (those contents are the string /B/Y).
• The size of A/New is 4 bytes, one for each character of its contents.
• If /B/Y is removed, /A/New becomes invalid.
• If a new /B/Y is created, A/New is once again valid.
• Removing /A/New has no effect of /B/Y.
• Examining /B/Y does not reveal the existence of /A/New.
• If a user has write permission for /B/Y, then writing /A/New is possible and writes /B/Y.

The bottom line is that, with a hard link, a new name is created for the file. This new name has equal status with the original name. This can cause some surprises (e.g., you create a link but I own the file). With a symbolic link a new file is created (owned by the creator naturally) that contains the name of the original file. We often say the new file points to the original file.

Question: Consider the hard link setup above. If Bob removes /B/Y and then creates another /B/Y, what happens to /A/New?
Answer: Nothing. /A/New is still a file owned by Bob having the same contents, creation time, etc. as the original /B/Y.

Question: What about with a symlink?
Answer: /A/New becomes invalid and then valid again, this time pointing to the new /B/Y. (It can't point to the old /B/Y as that is completely gone.)

Note:
Shortcuts in windows contain more than symlinks contain in unix. In addition to the file name of the original file, they can contain arguments to pass to the file if it is executable. So a shortcut to

    firefox.exe
  

    firefox.exe //cs.nyu.edu/~gottlieb/courses/os/class-notes.html
  

Moreover, as was pointed out by students in my 2006-07 fall class, the shortcuts are not a feature of the windows FAT file system itself, but simply the actions of the command interpreter when encountering a file named *.lnk
End of Note.

Symlinking a Directory

What happens if the target of the symlink is an existing directory? For example, consider the code below, which gives rise to the diagram on the right.

    cd /
    mkdir /A; mkdir /B
    touch /A/X; touch /B/Y
    ln -s /B /A/New
  

1. Is there a file named /A/New/Y ?
   Answer: Yes.
   
   
2. What happens if you execute cd /A/New; dir ?
   Answer: You see a listing of the files in /B, in this case the single file Y.
   
   
3. What happens if you execute cd /A/New/.. ?
   Answer: Not clear!
   Clearly you are changing directory to the parent directory of /A/New. But is that /A or /?
   The command interpreter I use offers both possibilities.
   • cd -L /A/New/.. takes you to A (L for logical).
   • cd -P /A/New/.. takes you to / (P for physical).
   • cd /A/New/.. takes you to A (logical is the default).
   
   
4. What did I mean when I said the pictures made it all clear?
   Answer: From the file system perspective it is clear. It is not always so clear what programs will do.