4.1.3: File types

Examples

1. (Regular) files.
   
   
2. Directories: studied below.
   
   
3. Special files (for devices). Uses the naming power of files to unify many actions.

       dir             # prints on screen
       dir > file      # result put in a file
       dir > /dev/tape # results written to tape
       

4. ``Symbolic'' Links (similar to ``shortcuts''): Also studied below.

``Magic number'': Identifies an executable file.

• There can be several different magic numbers for different types of executables.
• unix: #!/usr/bin/perl

Strongly typed files:

• The type of the file determines what you can do with the file.
• This make the easy and (hopefully) common case easier and, more importantly safer.
• It tends to make the unusual case harder. For example, you have a program that turns out data (.dat) files. But you want to use it to turn out a java file but the type of the output is data and cannot be easily converted to type java.

4.1.4: File access

There are basically two possibilities, sequential access and random access (a.k.a. direct access). Previously, files were declared to be sequential or random. Modern systems do not do this. Instead all files are random and optimizations are applied when the system dynamically determines that a file is (probably) being accessed sequentially.

1. With Sequential access the bytes (or records) are accessed in order (i.e., n-1, n, n+1, ...). Sequential access is the most common and gives the highest performance. For some devices (e.g. tapes) access ``must'' be sequential.
2. With random access, the bytes are accessed in any order. Thus each access must specify which bytes are desired.

4.1.5: File attributes

A laundry list of properties that can be specified for a file For example:

• hidden
• do not dump
• owner
• key length (for keyed files)

  4.1.6: File operations

  • Create: Essential if a system is to add files. Need not be a separate system call (can be merged with open).
  • Delete: Essential if a system is to delete files.
  • Open: Not essential. An optimization in which the translation from file name to disk locations is perform only once per file rather than once per access.
  • Close: Not essential. Free resources.
  • Read: Essential. Must specify filename, file location, number of bytes, and a buffer into which the data is to be placed. Several of these parameters can be set by other system calls and in many OS's they are.
  • Write: Essential if updates are to be supported. See read for parameters.
  • Seek: Not essential (could be in read/write). Specify the offset of the next (read or write) access to this file.
  • Get attributes: Essential if attributes are to be used.
  • Set attributes: Essential if attributes are to be user settable.
  • Rename: Tanenbaum has strange words. Copy and delete is not acceptable for big files. Moreover copy-delete is not atomic. Indeed link-delete is not atomic so even if link (discussed below) is provided, renaming a file adds functionality.

  Homework: 2, 3, 4.
  Read and understand ``copyfile'' on page 155.

  Notes on copyfile

  • Normally in unix one wouldn't call read and write directly.
  • Indeed, for copyfile, getchar() and putchar() would be nice since they take care of the buffering (standard I/O, stdio).
  • Tanenbaum is correct that the error reporting is atrocious.
    The worst is exiting the loop on error and thus generating an exit(0) as if nothing happened.

  4.1.7: Memory mapped files

  Conceptually simple and elegant. Associate a segment with each file and then normal memory operations take the place of I/O.

  Thus copyfile does not have fgetc/fputc (or read/write). Instead it is just like memcopy

  while ( (dest++)* = (src++)* );
  

  The implementation is via segmentation with demand paging but the backing store for the pages is the file itself. This all sounds great but ...

  1. How do you tell the length of a newly created file? You know which pages were written but not what words in those pages. So a file with one byte or 10, looks like a page.
  2. What if same file is accessed by both I/O and memory mapping.
     
  3. What if the file is bigger than the size of virtual memory (will not be a problem for systems built 3 years from now as all will have enormous virtual memory sizes).

  4.2: Directories

  Unit of organization.

  4.2.1: Hierarchical directory systems

  Possibilities

  • One directory in the system
  • One per user
  • One tree
  • One tree per user
  • One forest
  • One forest per user

  These are not as wildly different as they sound.

  • If the system only has one directory, but allows the character / in a file name. Then one could fake a tree by having a file named
    /allan/gottlieb/courses/arch/class-notes.html
    rather than a directory allan, a subdirectory gottlieb, ..., a file class-notes.html.
  • Dos (windows) is a forest, unix a tree. In dos there is no common parent of a:\ and c:\.
  • But windows explorer makes the dos forest look quite a bit like a tree. Indeed, the gnome file manager for linux, looks A LOT like windows explorer.
  • You can get an effect similar to (but not the same as) one X per user by having just one X in the system and having permissions that permits each user to visit only a subset. Of course if the system doesn't have permissions, this is not possible.
  • Today's systems have a tree per system or a forest per system.

  4.2.2: Path Names

  You can specify the location of a file in the file hierarchy by using either anabsolute versus or a Relative path to the file

  • An absolute path starts at the (or a if we have a forest) root.
  • A relative path starts at the current (a.k.a working) directory.
  • The special directories . and .. represent the current directory and the parent of the current directory respectively.

  Homework: 1, 8.

  4.2.3: Directory operations

  1. Create: Produces an ``empty'' directory. Normally the directory created actually contains . and .., so is not really empty
     
     
  2. Delete: Requires the directory to be empty (i.e., to just contain . and ..). Commands are normally written that will first empty the directory (except for . and ..) and then delete it. These commands make use of file and directory delete system calls.
     
     
  3. Opendir: Same as for files (creates a ``handle'')
     
     
  4. Closedir: Same as for files
     
     
  5. Readdir: In the old days (of unix) one could read directories as files so there was no special readdir (or opendir/closedir). It was believed that the uniform treatment would make programming (or at least system understanding) easier as there was less to learn.

     However, experience has taught that this was not a good idea since the structure of directories then becomes exposed. Early unix had a simple structure (and there was only one). Modern systems have more sophisticated structures and more importantly they are not fixed across implementations.
     
     

  6. Rename: As with files
     
     
  7. Link: Add a second name for a file; discussed below.
     
     
  8. Unlink: Remove a directory entry. This is how a file is deleted. But if there are many links and just one is unlinked, the file remains. Discussed in more detail below.

  4.3: File System Implementation

  4.3.1; Implementing Files

  • A disk cannot read or write a single word. Instead it can read or write a sector, which is often 512 bytes.
  • Disks are written in blocks whose size is a multiple of the sector size.
  • When we study I/O in the next chapter I will bring in some physically large (and hence old) disks so that we can see what they look like and understand better sectors (and tracks, and cylinders, and heads, etc.).

  Contiguous allocation

  • This is like OS/MVT.
  • The entire file is stored as one piece.
  • Simple and fast for access, but ...
  • Problem with growing files
    • Must either evict the file itself or the file it is bumping into.
    • Same problem with an OS/MVT kind of system if jobs grow.
  • Problem with external fragmentation.
  • Not used for general purpose systems. Ideal for systems where files do not change size.

  Homework: 7.

  Linked allocation

  • The directory entry contains a pointer to the first block of the file.
  • Each block contains a pointer to the next.
  • Horrible for random access.
  • Not used.

  FAT (file allocation table)

  • Used by dos and windows (but not windows/NT).
  • Directory entry points to first block (i.e. specifies the block number).
  • A FAT is maintained in memory having one (word) entry for each disk block. The entry for block N contains the block number of the next block in the same file as N.
  • This is linked but the links are store separately.
  • Time to access a random block is still is linear in size of file but now all the references are to this one table which is in memory. So it is bad but not horrible for random access.
  • Size of table is one word per disk block. If one writes all blocks of size 4K and uses 4-byte words, the table is one megabyte for each disk gigabyte. Large but not prohibitive.
  • If write blocks of size 512 bytes (the sector size of most disks) then the table is 8 megs per gig, which might be prohibitive.

  Inodes

  • Used by unix.
  • Directory entry points to inode (index-node).
  • Inode points to first few data blocks, often called direct blocks.
  • Inode also points to an indirect block, which points to disk blocks.
  • Inode also points to a double indirect, which points an indirect ...
  • For some implementations there are triple indirect as well.
  • The inode is in memory for open files. So references to direct blocks take just one I/O.
  • For big files most references require two I/Os (indirect + data).
  • For huge files most references require three I/Os (double indirect, indirect, and data).

  

  4.3.2; Implementing Directories

  Recall that a directory is a mapping that converts file (or subdirectory) names to the files (or subdirectories) themselves.

  Trivial File System (CP/M)

  • Only one directory in the system.
  • Directory entry contains pointers to disk blocks.
  • If need more blocks, get another directory entry.

  MS-DOS and Windows (FAT)

  • Subdirectories supported.
  • Directory entry contains metatdata such as date and size as well as pointer to first block.

  Unix

  • Each entry contains a name and a pointer to the corresponding inode.
  • Metadata is in the inode.
  • Early unix had limit of 14 character names.
  • Name field now is varying length.
  • To go down a level in directory takes two steps: get inode, get file (or subdirectory).
  • Do on the blackboard the steps for /allan/gottlieb/courses/os/class-notes.html

  Homework: 11