Operating Systems (V22.0202)
Homework 6
Miscellaneous Projects, Demand Paging

Due: Dec 18, 2006

Introduction

Generally speaking, you should begin your work starting from your solution to hw5. In particular, we assume that SOS/STM has the ability to do string I/O, supports multiprocessing, and has simple process synchronization. If you like, you can begin with our files stm.h, stm.c, sos.h, sos.c (but note that these lack the synchronization primitives).

In this document, I will first mention several possible projects, of varying degrees of difficulty, essentially independent of each other. Then I will go into detail one particular project, which is to introduce demand paging into SOS.

In this project, you are allowed to work in pairs (you would have to work hard to convince me if you want to work in groups larger than 2. Working alone is also fine.

CVS Tool: In this course, we emphasize two tools used by programmers: the C language and Makefile. Whether you work alone or in pairs, you have the opportunity to learn another important system building tool, "version control systems". These are software that help us manage large software projects that are worked on by many (or even one) programmer. We will like to use the "Concurrent Version System" (CVS) which is freely available on many platforms, including CYGWIN. Such systems allows you to keep track of different versions of a file, and even develop branching versions. Different users can simultaneously modify a file, since the system has a method of reconciling differences (which is quite amazing if you first see this at work).

If you want to use CVS, please see my web page on how to use CVS. Send us your pub_id.dsa file, and you and your partner will be ready to share files in this project.

Possible Projects

1. ENHANCED ASSEMBLER:
   Enhance the assembler (assem.pl) (we recommend to continue using a perl script). The assembler is a critical tool that will be useful for the larger projects (e.g., our next project below). For instance, we would like to allow the assembler language to support

   	LOA R15,=1
   	

   instead of having to explicitly load the constant "1" into a location, and write the address of that location into our instruction for this load instruction. Also, we would like to be able to explicitly enter a literal constant at the current location: e.g.,

   	CON 123
   	CON "Hello"
   	

   should load the CONstant 123, followed by the ASCII codes for "H", "e", "l", "l", "o" into consecutive locations. Similarly, we should be able to reserve a block of memory: e.g.,

   	RES 100
   	

   will REServe the next block of 100 words of memory. The directory for the original assembler is HERE.
2. RESIDENT SOS:
   Rewrite SOS so that it is an stm program. In short, we want a stm program called "sos.stm". When you start up the stm simulator, it begins by loading sos.stm and begin executing this! Thus, the OS in now resident stm memory, as in any real machines. An enhanced assembler would help us in this task. This project would be quite big if we want to implement multi-processing. But doing a uni-process sos should be relatively easy. This is basically a simple loop.
3. SIMPLE FILE SYSTEM:
   Implement a simple file system (SFS) that has only one directory, and which supports only ASCII files. You need to implement the basic file functions: open, close, create, delete, read, seek, write. Initially, you may only implement "open, read, close". This would already be useful for our some of our other projects. For instance, SOS can now load stm programs from files into memory to be executed. We rely on other means for creating the actual files.
4. LIBRARY SYSTEM:
   Unix has over 300 such library routines to carry out many of its basic functions. These routines provide an easy way to extend the functionality of the basic Kernel. We want to duplicate this in SOS.

   Implement the ability of stm programs to call system library routines. The ability of SOS to open/read/close files, as described in the previous project, would be useful, but otherwise, we can simulate this. This requires some primitive loader software. Write some useful library be able to write useful library routines to provide more flexible I/O (the equivalent of scanf and printf).

5. SIMPLE SHELL (SS):
   We want a simple interface (SS) for interacting with SOS. This interface to sit in a while loop waiting for user commands. This is the beginning of a primitive SHELL (sh/bash/csh/tsh/ksh). In this loop, the SS can accept these commands:

   SOS commands

   Command	Arguments	Action
   jobs	[-q quantum][-d debug_level][-m max] prog1.stm prog2.stm ...	similar to arguments for SOS in hw4
   status	[no arguments]	shows the list of jobs currently running
   halt	[no arguments]	shuts down the stm hardware

   When we start up the SOS, you call "SOS [initfile]" where initfile is a file that contains various initialization values. This will (load the OS) and call SS.

   What are some parameter values that can go into the initfile? The file [initfile] is optional (or has default values). Run time parameters such as Quantum=5, DebugLevel=0, PageSize=512, UserSpaceStart=16K, etc.

   We should divide the memory into Kernel Space and User Space. UserSpaceStart is the starting address for User Space (so Kernel Space is mem[0] to mem[UserSpaceStart-1]).

6. DEMAND PAGING:
   See next.

Demand Paging System

We can tell you more about what is wrong: when we want to replace a page with mbit=1 (modify bit is set), there must be write-to-disk, followed by a read-from-disk. It seems that this program forgets to the write-to-disk.

In the following, we try to outline the logic found in the buggy file. To do paging, you need a secondary memory. So, in addition to main memory (mem) we have another array called disk.

DETAILS

• You will need to define some constants. Here are some typical numbers:

  	#define VASIZE 262144	   	/* 2^18 virtual address space */
  	#define RAMSIZE  65536     	/* 2^16 main memory */
  	#define DISKSIZE  2097152  	/* 2^22 disk memory */
  	#define PAGESIZE 512   	   	/* 2*9 page size */
  	#define MAXFRAMES (RAMSIZE / PAGESIZE)   /* frame table size */
  	#define MAXPAGES (DISKSIZE / PAGESIZE)   /* page table size */
  	

• We need to declare the process table, and for each process, we need to declare a page table. There is also a page table for the current process:

  	struct pageTableEntry {
  	  int frame,               /* frame number (frame=-1 means not loaded)*/
  	      disk,                /* relative page number in disk */
  	      rbit,
  	      mbit,
  	      timestamp;
  	}
  	
  	struct processTableEntry {
  	  char name[NAMESIZE];
  	  int regs[NUMREGS];
  	  int status;     	/* 0=ready, 1=blocked, 2=terminated */
  	  struct pageTableEntry pageTable[MAXPAGES];  /* page table of process*/
  	  int basePage;         /* first page in disk for process */
  	  int pageLimit;        /* max number of disk page for process */
  	}
  
  	struct processTableEntry processTable[MAXPROCS];
  
  	struct pageTableEntry pageTable[MAXPAGES];
  				/* page table of current process*/
  	

  You would want procedures to initialize a pageTable, and to load a pageTable from the processTable into the current pageTable. (For simplicity, you do not hav eto put this table into the mem[].)
• We need a FIFO called readyQueue (holding the processes ready to run).
• To keep track of the available frames, we have

  	freeList[MAXFRAMES];	/* freeList[n]=1 is occupied, 0 if free */
  	

  Assume a routine to find a free frame (return -1 if no more free frames).
• We need some global variables

  	int disk[DISKSIZE];	/* disk */
  	int pageFault = -1;	/* if pageFault is not -1, it is the
  				   page that must be loaded to memory */
  	int clock;		/* system clock */
  	

  You need procedure to initialize the disk, and to load a process into the disk.
• To simulate asynchronous disk and memory I/O, we have a list of disk requests. Each request transfers a single page between disk and memory. We can assume that each process make at most one disk request at a time, and so this list can be a table with MAXPROCS entries.

  	struct diskRequest {
  	   int diskPage;   	/* page number in disk */
  	   int memPage;   	/* frame number in main memory */
  	   int write;   	/* write=1 if writing, write=0 if reading */
  	}
  
  	struct diskRequest RequestTable[MAXPROCS];
  				/* RequestTable[n] is request of Process n */
  	

  You need functions to (1) post a disk request, (2) process a disk request, (3) read from Disk, and (4) write to Disk.
• Here is the algorithm to convert a relative address to a physical (memory) address.

  	int physAddr( int relAddr, int writeBit);
  
  	NOTE: writeBit=1 if we want to write, writeBit=0 if we want to read
  
  	1. Make sure that relAddr is between 0 and VASIZE.  Else segment fault.
  
  	2. Initialize pageFault=-1.   (This will change if we need a page fault)
  
  	3. page = ralAddr / PAGESIZE
  
  	4. frame = pageTable[page].frame
  
  	5. Update rbit, mbit, timestamp of pageTable[page]. (N.B. mbit=writeBit)
  
  	6. if (frame = -1)
  	         pageFault = page;     (indicate need for page fault)
  	   else
  	   	return ((frame * PAGESIZE) + (relAddr % PAGESIZE))
  	

• How do we use this physAddr(..)? In the exec_instr(..) function! Recall that function has a switch instruction:

  
  	exec_instr(op, ra, ad, ...) {
  
  	    ...
  
  	    switch (op) {
  	       case 0:  pa = physAddr(ad, 0);        /* 0=read request */
  	             	if (pageFault == -1) 
  				regs[ra] = mem[pa];  /* load register ra */
  			break;
  	       case 1: ...
  
  		...
  
  	    }//switch
  
  	    ...
  
  	}//exec_instr
  
  	

  Note that in case of page fault, the instruction is not carried out. That is because the current process would be context switched anyway, and block while waiting for disk I/O.
• Let us now consider the page fault algorithm:

  	void execPageFault () {
  
  		1. frame = freeFrame();    // try to get a free frame
  
  		2. if (frame != -1)        // found free frame!
  		        Update page table
  			Return
  		3. Find a frame to evict   // this is the victim frame
  
  		4. If the mbit of victim is set,
  
  			post a disk request to write 
  
  			post a disk request to read 
  
  			contextSwitch( 1 )   // 1=block current process
  
  		   else
  
  		   	post a disk request to read
  
  			contextSwitch( 0 )   // 0=nonblock current process
  	}
  	

SUBMITTING YOUR WORK

• Your submission must include a README file telling us (1) what you have implemented and (2) what you have not implemented (but would like to). (3) It should also give a high level description of how you implemented various features, e.g., the key data structures and important functions and what they do.
• There should be a Makefile with the proper targets. For Demand Paging, the targets should be "stm" to compile the stm, "sos" to compile the SOS, and "test" to show what your sos can do.
• Remember the general rule: if it does not compile and work on simple tests, we cannot give any credit! So, it is more important to get your program to work on SOME easy features than to try to cover ALL the features in your project.

ADDITIONAL COMMENTS

• Another feature (future project) is to implement page caching on top of demand paging. A standard hardware page cache is the TLB (Translation Look-Aside Buffer) that can hold up to 64 entries of the page table. To verify the effectiveness of this cache, we can conduct some experiments: to use page sizes of 8 (!) words, and run GCD on huge Fibonacci Numbers as well as huge primes.