Operating Systems

Start Lecture #12

4.4.3 File System Consistency

Modern systems have utility programs that check the consistency of a file system. A different utility is needed for each file system type in the system, but a wrapper program is often created so that the user is unaware of the different utilities.

The unix utility is called fsck (file system check) and the window utility is called chkdsk (check disk).

• If the system crashed, it is possible that not all metadata was written to disk. As a result the file system may be inconsistent. These programs check, and often correct, inconsistencies.
• Scan all i-nodes (or fat) to check that each block is in exactly one file, or on the free list, but not both.
• Also check that the number of links to each file (part of the metadata in the file's i-node) is correct (by looking at all directories).
• Other checks as well.
• Offers to fix the errors found (for most errors).

Bad blocks on disks

Not so much of a problem now. Disks are more reliable and, more importantly, disks and disk controllers take care most bad blocks themselves.

4.4.4 File System Performance

Caching

Demand paging again!

Demand paging is a form of caching: Conceptually, the process resides on disk (the big and slow medium) and only a portion of the process (hopefully a small portion that is heavily access) resides in memory (the small and fast medium).

The same idea can be applied to files. The file resides on disk but a portion is kept in memory. The area in memory used to for those file blocks is called the buffer cache or block cache.

Some form of LRU replacement is used.

The buffer cache is clearly good and simple for reads.

What about writes?

• Must update the buffer cache (otherwise subsequent reads will return the old value).
• The major question is whether the system should also update the disk block.
• The simplest alternative is write through in which each write is performed at the disk before it declared complete.
  • Since floppy disk drivers adopt a write through policy, one can remove a floppy as soon as an operation is complete.
  • Write through results in heavy I/O write traffic.
    • If a block is written many times all the writes are sent the disk. Only the last one was needed.
    • If a temporary file is created, written, read, and deleted, all the disk writes were wasted.
  • DOS uses write-through. Windows now uses write-back for hard drives.
• The other alternative is write back in which the disk is not updated until the in-memory copy is evicted (i.e., at replacement time).
  • Much less write traffic than write through.
  • Trouble if a crash occurs.
  • Used by Unix and others for hard disks.
  • Can write dirty blocks periodically, say every minute. This limits the possible damage, but also the possible gain.
  • Ordered writes. Do not write a block containing pointers until the block pointed to has been written. Especially if the block pointed to contains pointers since the version of these pointers on disk may be wrong and you are giving a file pointers to some random blocks.

Homework: 27.

Block Read Ahead

When the access pattern looks sequential read ahead is employed. This means that after completing a read() request for block n of a file. The system guesses that a read() request for block n+1 will shortly be issued so it automatically fetches block n+1.

• How does the system decide that the access pattern looks sequential?
  • If a seek system call is issued, the access pattern is not sequential.
  • If a process issues consecutive read() system calls for block n-1 and then n, the access patters is guessed to be sequential.
  
  
• What if block n+1 is already in the block cache?
  Ans: Don't issue the read ahead.
  
  
• Would it be reasonable to read ahead two or three blocks?
  Ans: Yes.
  
  
• Would it be reasonable to read ahead the entire file?
  Ans: No, it could easily pollute the cache evicting needed blocks, and could waste considerable disk bandwidth.

Reducing Disk Arm Motion

Try to place near each other blocks that are going to be accessed sequentially.

1. If the system uses a bitmap for the free list, it can allocate a new block for a file close to the previous block (guessing that the file will be accessed sequentially).
   
   
2. The system can perform allocations in super-blocks, consisting of several contiguous blocks.
   • The block cache and I/O requests are still in blocks not super-blocks.
   • If the file is accessed sequentially, consecutive blocks of a super-block will be accessed in sequence and these are contiguous on the disk.
   
   
3. For a unix-like file system, the i-nodes can be placed in the middle of the disk, instead of at one end, to reduce the seek time to access an i-node followed by a block of the file.
   
   
4. The system can logically divide the disk into cylinder groups, each of which is a consecutive group of cylinders.
   • Each cylinder group has its own free list and, for a unix-like file system, its own space for i-nodes.
   • If possible, the blocks for a file are allocated in the same cylinder group as the i-node.
   • This reduces seek time if consecutive accesses are for the same file.

4.5 Example File Systems

4.5.A The CP/M File System

CP/M was a very early and simple OS. It ran on primitive hardware with very little ram and disk space. CP/M had only one directory in the entire system. The directory entry for a file contained pointers to the disk blocks of the file. If the file contained more blocks than could fit in a directory entry, a second entry was used.

4.5.1 CD-ROM File Systems

File systems on cdroms do not need to support file addition or deletion and as a result have no need for free blocks. CD-R (recordable) do permit files to be added, but they are always added at the end of the disk. The space allocated to a file is not recovered even when the file is deleted so the free list is simply the blocks after the last file recorded.

The result is that the file systems for these devices are quite simple.

The ISO9660 File System

The basic file system for essentially all data cdroms (music cdroms are different and are not discussed). Most Unix systems use iso9660 with the Rock Ridge extensions and most windows systems use iso9660 with the joliet extensions.

The standard permits a single physical cd to be partitioned and permits a cdrom file system to span many physical cd. However, these features are rarely used and we will not discuss them.

Since files do not change each directory entry need only give the starting location and length of a file.

File names are 8+3 characters (directory names just 8) for iso9660-level-1 and 31 characters for -level-2. There is also a -level-3 in which a file is composed of extents which can be shared among files and even with a single file (i.e. a single physical extent can occur multiple times in a given file).

Directories can be nested only 8 deep.

Rock Ridge Extensions

The Rock Ridge extensions were designed by a committee from the unix community to permit a unix file system to be copied to a cdrom without information loss.

These extensions included.

1. The unix rwx bits for permissions.
2. Major and Minor numbers to support special files, i.e. including devices in the file system name structure.
3. Symbolic links.
4. An alternate (long) name for files and directories.
5. A somewhat kludgy work around for the limit nesting levels.
6. Unix timestamps (creation, last access, last modification).

Joliet Extensions

The Joliet extensions were designed by Microsoft to permit a windows file system to be copied to a cdrom without information loss.

These extensions included.

1. Long file names.
2. Unicode.
3. Arbitrary depth of directory nesting.
4. Directory names with extensions.

4.4.3 The MS-DOS File System

MS-DOS and Windows (FAT)

This file system has been supported since the first IBM PC (1981) and is still used. Indeed considering the number of cameras and MP3 players, it is very widely used.

Unlike CP/M, MS-DOS always had support for subdirectories and metadata such as date and size.

File names were restricted to 8+3.

As described above, the directory entries pointed to the first block of the file and the FAT has the pointers to the remaining blocks.

The free list was supported by using a special code in the FAT for free blocks.

The first version FAT-12 used 12-bit block numbers so a partition could not exceed 2¹² blocks. A subsequent release went to FAT-16.

4.4.4 The Windows 98 File System

Two changes were made: Long file names were supported and the file allocation table was switched from FAT-16 to FAT-32. These changes first appeared in the second release of Windows 95.

The hard part was keeping compatibility with the old 8+3 naming rule. That is new file systems created with windows 98 using long file names must be accessible if the file system is used with an older version of windows that supported only 8+3 file names. This is called backwards compatibility.

A file has two names a long one and an 8+3 one. The primary directory entry for a file in windows 98 is the same format as it was in MS-DOS and contains the 8+3 file. If the long name fits the 8+3 format, the story ends here.

If the long name does not fit the 8+3, an 8+3 version is produce via an algorithm, that works but produces names with severely limited aesthetic value. The long name is stored in one or more axillary directory entries directly before the main entry. These axillary entries are set up to appear invalid to the old OS, which (happily?) ignores them.

FAT-32 used 32 bit words for the block numbers (actually, it used 28 bits) so the FAT could be huge. Windows 98 kept only a portion of the FAT-32 table in memory at a time. (I do not know the replacement policy, number of blocks kept in memory, etc).

4.4.5 The Unix V7 File System

We presented the i-node system in some detail above. Here we add a few more points.

• Each directory entry contains a name and a pointer to the corresponding i-node.
• The metadata for files and directories in the corresponding i-nodes.
• Early unix limited file names to 14 characters, stored in a fixed length field.
• The name field now is of varying length and file names can be quite long. On my linux system
      touch 200-char-name
  was OK but
      touch 300-char-name
  was not. I suppose the limit is around 256, but didn't test that.
• To go down a level in the directory hierarchy takes two steps: get i-node, get file (or subdirectory).
• This shows how important it is not to parse filenames for each I/O operation, i.e., why the open() system call is important.
• Do on the blackboard the steps for /a/b/X

4.6 Summary (read)