CS372H Spring 2010 Homework 8 Solutions

Problem 1:

Consider a very simple file system for a tiny disk. Each sector on the disk holds 2 integers, and all data blocks, indirect blocks, and inodes are 1 disk sector in size (each contains 2 integers). All files stored on disk are interpreted as directories by the file system (there are no "data files"). The file system defines the layout for the following data types on disk:
inode = 1 pointer to a data block + 1 pointer to indirect block

indirect block = 2 pointers to data blocks

directory = a regular file containing zero or more pairs of integers; the first integer of each pair is a file name and the second is the file's inumber

The value "99" signifies a null pointer when referring to a disk block address or directory name. An empty directory has one disk block with the contents "99   99". The inumber for root directory is "/" is 0.

The following data are stored on disk:

inode array:

0 1  2  3  4   5  6  7   8   9  10 11 12 13 14 15
10
6		 7
99		    8
99		       3
99		                           

disk blocks:

 0  1  2  3  4   5  6 7 8  9  10 11 12 13 14 15
   32
3		    96
1		       1
99		 99
99		 99
99		    57
6		               

  1. How many entries can appear in a maximum-sized directory? (Each entry is a pair of integers)

    Solution: 3 ( one data block and two other data blocks pointed by the one indirect block)

  2. List all directories stored on this disk (full path names) along with the names of the files stored in each directory.

    Solution:

    directory path name inumber indirect blocks data blocks contents (subdirectories)

    /

    0

    6

    		  10 1  /32,  /57

    /32

    3

    n/a

    		  8  n/a

    /57

    6

    n/a

    		  3  /57/96

    /57/96

    1

    n/a

    		  7  n/a
            

  3. Modify the above data structures to add an empty directory called "87" to directory "/"

    Solution:

    directory path name inumber indirect blocks data blocks contents (subdirectories)

    /87

    9

    n/a

    		 14 n/a

    /

    0

    6

    		  10  1  13  /32,  /57, /87

     

    0 1  2  3  4   5  6  7   8   9  10 11 12 13 14 15
    10
    6    		 7
    99    		    8
    99    		       3
    99    		       14
    99          		                  

    disk blocks:

     0  1  2  3  4   5  6 7 8  9  10 11 12 13 14 15
       32
    3    		    96
    1    		       1
    13    		 99
    99    		 99
    99    		    57
    6    		       87
      9    		 99
    99          		   

Problem 2:

Is it fundamentally necessary to store on disk the information about the unallocated disk sectors? Explain why.

Solution:
No. This information can be computed at startup by traversing the file system tree. However, with this information stored on the disk, the job of a utility like fsck is simplified.

Problem 3:

The FastFile file system uses an inode array to organize the files on disk. Each inode consists of a user id (2 bytes), three time stamps (4 bytes each), protection bits (2 bytes), a reference count (2 byte), a file type (2 bytes) and the size (4 bytes). Additionally, the inode contains 13 direct indexes, 1 index to a 1st-level index table, 1 index to a 2nd-level index table, and 1 index to a 3rd level index table. The file system also stores the first 436 bytes of each file in the inode.
  1. Assume a disk sector is 512 bytes, and assume that any auxilliary index table takes up an entire sector, what is the maximum size for a file in this system.
  2. Is there any benefit for including the first 436 bytes of the file in the inode?

Solution:

  1. Note: The first thing you need to determine is the size of an index (2 bytes or 4 bytes). Given that we have a 3-level indexing scheme, you can quickly compute the number of sectors that you can get by 2 byte index and 4 bytes indexes. You will see that the 2-byte indexing does not work. This would give you up to 256 indexes per sector, with a maximum file size of 436 + 13 * 512 + 1 * 256 * 512 + 1 * 256 * 256 * 512 + 1 * 256 * 256 * 256 * 512 = whatever. The problem with this analysis is that you have far more disk sector for this scheme to work than can be encoded in 2 bytes. You can use 3 bytes, but this can get ugly. So, we go with 4-byte indexes, giving us 128 indexes per sector, and the correct answer is: 436 + 13 * 512 + 1 * 128 * 512 + 1 * 128 * 128 * 512 + 1 * 128 * 128 * 128 * 512 = 1082203060, roughly 1G.
  2. Yes. Most files are small. If the file size is 436 bytes or less, then the entire file can be read and written in one disk operation without having to do a separate access to the inode.

Problem 4:

Can we implement hard links in DOS (which uses a FAT approach)? Why or why not?

Solution:
No, we cannot. The reason is that the name space in DOS is fused with the file data structures unlike in UNIX, where the name space is in a directory structure that is separate from the file data structures (inode). See the lectures note for more detail.

Problem 5:

Can we implement symbolic links in DOS (FAT file system)? If so, show how, and if not, explain why.

Solution:
Yes. It can be done by assigning a bit in the directory to indicate that this entry is a symbolic name. Then, the name is stored as part of the file data of the symbolic link name. The operating system must then be modified to recognize the bit that expresses whether this is a symbolic link, and therefore perform an extra search during lookups.

Problem 6:

In some early releases of an operating system that shall remain nameless, when a file was deleted, its sectors reverted to the free list but they were not erased. What problems do you think may result from this? Why do you think the blocks were not erased?

Solution:
The problem is that if these free blocks are later allocated to a file, there is a potential that the blocks may later be allocated to a file belonging to a different user. If the file system allows such blocks to be read by another process (for example, in a system that does not maintain the size of the file in bytes, or in a system that allows files to be mapped to main memory), then a different user can read the data that belonged to another user.

Problem 7:

Pooh Software Ltd. is selling a file system that uses a UNIX-like file system with multi-level indexing. For more reliability, the inode array is actually replicated on the disk in two different places. The intent is that if one or a group of sectors that are storing either replica of the array become bad, the system can always recover from the replica. Discuss the effect of having this replicated data structure on performance.

Solution:
Updates to inodes will have to be done to both copies. This will decrease the performance of operations that attempt to modify the inodes, such as allocating a new file, deleting a file, extending the size of a file, or opening a file for updates (among perhaps many others). However, reading information from an inode can be made faster with clever disk scheduling. To read a particular inode, we schedule a read from either track that is closest to the current head position.

Problem 8:

Consider an indexed file allocation using index nodes (inodes). An inode contains among other things, 7 indexes, one indirect index, one double index, and one triple index.
  1. What usually is stored in the inode in addition to the indexes?
  2. What is the disadvantage of storing the file name in the inode? Where should the file name be stored?
  3. If the disk sector is 512 bytes, what is the maximum file size in this allocation scheme?
  4. Suppose we would like to enhance the file system by supporting versioning. That is, when a file is updated, the system creates new version leaving the previous one intact. How would you modify the inode allocation scheme to support versioning? You answer should consider how a new version is created and deleted.
  5. In a file system supporting versioning, would you put information about the version number in the inode, or in the directory tree? Justify your answer.

Solution:

  1. An inode usually stores indexes and:
  2. Storing the file name in the inode limits the flexibility of the file system and precludes the use of hard links. Also, since it is desirable to have relatively long file names, it would be cumbersome to store variable size character arrays in the inode structure (or wasteful if a certain maximum is defined).
  3. We must first determine the size of an index. For a 2-byte index, we can have 65536 disk blocks, i.e. 512 * 65536 = 32MB. But a triple index structure can express more disk blocks. Therefore, we go to a 4-byte indexing scheme (3-byte indexing scheme are not attractive and are not sufficient). A 4-byte indexing scheme therefore gives a maximum file size of 7*512 + 128*512 + 128*128*512 + 128*128*128*512 = 108219952 or about 1Gbytes
  4. We augment the inode structure such that different versions of the file share the common blocks. When a new version is created, we copy the information from the older version into the new one. Also, we will need to add a reference count to every disk block, and set that reference count to 1 when a block is allocated to a single file. The reference count is incremented whenever a block is shared among different versions. Then, when a version is modified, we perform a copy-on-write like policy and decrement the reference count of the disk blocks and actually copy the modified disk blocks and make them only accessible to the new version.
  5. It is better put in the directory space in order to make it easier for the user to relate the different versions of the same file.

Problem 9:

The LoneStar backup system for UNIX works as follows: At the beginning of each week, the backup system traverses the file system tree structure and saves all directories and files. This stage is called full backup. Then, on a daily basis, it performs incremental backups. It does so by traversing the directory tree, and saving only the files whose time stamps show that they have been modified since the previous incremental backup. Note that the LoneStar system does not follow the symbolic links when saving files, instead storing them as symbolic links. This preserves the integrity of the symbolic link.
  1. What happens if the backup system were to follow the symbolic links and save the files the link points to?
  2. Identify problems with this backup scheme.
  3. Instead of doing the above, the KeyStone backup system requires the file system to maintain a bit map for every sector on disk. If a disk sector is updated, the file system sets the corresponding bit to one. When the KeyStone system performs an incremental backup, it saves only the disk sectors whose bits are set, then it resets the entire bit map. Compare between the two backup schemes. What are  the common problems between them? What are the key advantages of each? Which one would you use?

Solution:

  1. That would be unfortunate, because should the file is restored from the backup version, it will no longer be a symbolic link. Instead, it will be the file that happened to be pointed to by the link at the time of the backup, which is not semantically correct. For correct operation, the backup system must restore a symbolic link.
  2. For large files, it is sufficient that a single byte be modified to have the backup system save the entire file. Ie., the backup system is not smart about trying to reduce the amount of space that need to be saved (it saves the entire file if it has been modified, instead of just saving the difference between the old and new file). However, this is not a serious problem.
  3. The main difference between the two backup systems is that the KeyStone operates at the level of the flat file system, while the LoneStar system operates at the level of the name space. The KeyStone may look smarter in that it will avoid saving disk blocks that have not been modified, but it has several serious problems. First, it requires the filesystem to manipulate the bitmap, which adds serious overhead to day-to-day operation. Second, since it operates at the level of the flat-file system, it becomes problematic when individual files are to be restored to a different file system. Third, it may still save disk blocks that have been modified but are currently on the free list (if they have been modified then deallocated).

Problem 10:

Explain what happens to the disk in UNIX when a user in a text editor saves a new file called "foo" into the current directory. Assume that foo is 15678 bytes in length, that a disk block is 1024 bytes, that the file system supports up to 2^32 sectors, and that an inode contains 10 direct block pointers, 1 indirect block pointer, 1 doubly-indirect block pointer, and 1 triply-indirect block pointer. The system uses on-disk free maps for tracking free blocks and free inodes. Here's the set of actions that you may assume that the editor makes:
fd = open(foo, O_CREAT|O_WRONLY);
p = &editBuffer;
numBlocks = editBufferSize / 1024;
for (i = 0; i < numBlocks; i++) {
write(fd, p, 1024);
    p += 1024;
}
lengthOfLastPartialBlock = editBufferSize % 1024;
write(fd, p, lengthOfLastPartialBlock);
close(fd);
Assume that creating a file is synchronous -- all writes to disk complete before the call returns. Assume that write() calls are asynchronous (they write data and metadata to memory, but don't force the writes to disk), but that close() does not return until all writes to the file and metadata are safely on disk. Assume that all of the data structures that must be read to perform this action are already in the cache.
  1. Assuming the system does not use logging, explain what blocks get written out to the disk and what each block contains.

    Solution:

    // fd = open(foo, O_CREAT|O_WRONLY) -- this creates the file
    // suppose inode N is initially free and is allocated for this file
    write free inode map (mark entry N as "used" to allocate the inode)
    write inode N (all pointers are "NULL" -- 0-length file)
    write file block for directory containing "foo" by adding entry "foo -> N" (assume that this does not increase the number of blocks in that directory)

    // 16 write() calls cause no disk accesses

    // close
    write updates to free block map to allocate 17 blocks -- 16 data blocks + 1 indirect block
    write inode containing pointer to 10 data blocks and the indirect block and marking length of file as 15678 bytes
    write indirect block containing pointer to 6 data blocks
    write 16 data blocks

  2. Suppose the machine crashes after writing the inode, free map, and data blocks, but before writing any indirect blocks. Describe how, if not fixed when the system reboots, such an inconsistency could cause security violations in which one user could access the files of another user (be specific: How would this happen? Which blocks are vulnerable? Can any of the user's data be exposed to other users? Can any other user's data be exposed to this user?)

    Solution:
    In this situation, the inode points to an indirect block that contains garbage. This means that it could contain pointers to blocks allocated to other files (and other users). The user will be able to read up to 6 blocks of other users (reading the last 6 blocks of the file will follow these bogus pointers.) Other users may also read this user's data (future writes by this user to those blocks will follow these bad pointers and puts data into other users' files; this data may now be read by other users.)

  3. Suppose you were to implement write-ahead logging to fix this problem. Describe exactly what your system would write to the log to satisfy this user's requests, when the writes occur, and when the system can "apply" the updates -- copy them to their normal positions on disk.

    Solution:
    A simple answer is to make the open correspond to a "transaction begin" and the close correspond to a "transaction end". Then all of the above writes will first go to the log, then the close would also write a "commit" to the log. Then all writes in the log can be appplied.

Problem 11:

  1. An engineer has designed a FAT-like system and he has used 24 bits for each entry. For a 32-GB disk, what is the minimum size of a file allocation in this system? Justify your answer.
    Solution:
    A 32 GB disk has 2^35 bytes of storage; if each entry in the FAT has 24 bits, then there can be at most 2^24 allocation chunks (one per FAT entry), so each allocation chunk must be 2^11 bytes = 2KB

  2. Consider an index-based file system with the inode containing 64 indexes, 1 indirect index pointing to a disk block containing an array of direct indexes, and 1 2-level index in the usual way. Assume that each index takes 4 bytes.
    1. What is the maximum file size under this arrangement, if a disk block is 1024 bytes? Explain how do you compute this maximum size.
      Solution:
      An indirect block has 1024 bytes * 1 index/4 bytes = 256 pointers
      So a file has at most 64 + 256 + 256*2 blocks and (64 + 256 * 256^2) * 1024 bytes.

    2. How many disk accesses does it take to read one disk block at location 3000321 within a file, assuming no caching. Justify your answer.
      Solution:
      Block 3000321 is block number 2930, which means we have to read the inode, the 2-level index, a direct index, and the data block. 4 reads.

Problem 12:

Versioning: It is often desirable for users to maintain different versions of the same file (for example, during program development, for recovering in the case of wrong updates applied to a version, etc.). The VMS file system implements versioning by creating a new copy for the file every time it is opened for writing. The system also supported versioning in the naming structure, appending a version number to the name of each file. For instance, when foo.c;1 is updated, the system creates foo.c;2, etc. The system implemented commands to manipulate versions, for example, the PURGE command would delete all versions except the most recent one. Also, users could disable versioning for some files. Discuss the pros and cons for this scheme.

Solution:
The main advantage of this scheme is its simplicity. There really is no support that is fundamentally needed from the operating system. Instead, application programs and the shell can be all linked with a modified library that creates the new version whenever a file is open for update. The creation of the file can be done with the available system calls. For example, in a UNIX-like system this can be implemented by having the open() library stub actually calls creat() with the new name, then do the copy, then open the newly created file.

The obvious disadvantage of this scheme is performance and overhead. Opening a file to update a single byte would have to make an entire file copy. Also, there is an overhead in storage in that even though multiple versions of the same file have common information, they may not attempt to share the disk blocks that contain the common information. This results in a waste in space. However, such sharing is possible only with non-trivial modifications to the file system.

Problem 13:

An implementation of the FSCK program traverses the file system tree and builds two lists of the disk blocks. One list contains the sectors that are shown to be in use, while the other reads the free disk-block information on disk. If the two lists for 4 blocks were as shown:

In use 0 1 0 1
Free 0 0 1 1

Identify what are the problems, if any, and what should fsck do for each block.

Solution:
The (0, 0) combination is a problem because it shows the disk block not belonging to any file, and yet it is not on the free list. Fsck should mark the block as used and attempts to find where it should belong to. If it fails, it should create a recovered file in the lost+found directory and includes that block there.

The (1,1) combination is a problem because it shows the disk block pertaining to a file an yet also on the free list. Fsck should mark the block as used and keep the block to the file where it belongs.