Administrivia

Web Pages

There is a web page for the course. You can find it from my home page.

• Can find these notes there. Let me know if you can't find it.
• They will be updated as bugs are found.
• Will also have each lecture available as a separate page. I will produce the page after the lecture is given. These individual pages might not get updated.

Textbook

Text is Tanenbaum, "Modern Operating Systems".

• Available in bookstore.
• We will do part 1, starting with chapter 1.

A Grade of ``Incomplete''

It is university policy that a student's request for an incomplete be granted only in exceptional circumstances and only if applied for in advance. Of course the application must be before the final exam.

Computer Accounts and Mailman mailing list

• You are entitled to a computer account, get it.
• Sign up for a Mailman mailing list for the course. http://www.cs.nyu.edu/mailman/listinfo/g22_2250_001_fl00
• If you want to send mail to me, use gottlieb@nyu.edu not the mailing list.
• You may do assignments on any system you wish, but ...
  • You are responsible for the machine. I extend deadlines if the nyu machines are down, not if yours are.
  • Be sure to upload your assignments to the nyu systems.
    • If somehow your assignment is misplaced by me or a grader, we need a to have a copy ON AN NYU SYSTEM that can be used to verify the date the lab was completed.
    • When you complete a lab (and have it on an nyu system), do not edit those files. Indeed, put the lab in a separate directory and keep out of the directory. You do not want to alter the dates.

Homework and Labs

I make a distinction between homework and labs.

Labs are

• Required
• Due several lectures later (date given on assignment)
• Graded and form part of your final grade
• Penalized for lateness

• Optional
• Due the beginning of Next lecture
• Not accepted late
• Mostly from the book
• Collected and returned
• Can help, but not hurt, your grade

Upper left board for assignments and announcements.

Interlude on Linkers

Originally called linkage editors by IBM.

This is an example of a utility program included with an operating system distribution. Like a compiler, it is not part of the operating system per se, i.e. it does not run in supervisor mode. Unlike a compiler it is OS dependent (what object/load file format is used) and is not (normally) language dependent.

What does a Linker Do?

Link of course.

When the assembler has finished it produces an object module that is almost runnable. There are two primary problems that must be solved for the object module to be runnable. Both are involved with linking (that word, again) together multiple object modules.

1. Relocating relative addresses.

   

   • Each module (mistakenly) believes it will be loaded at location zero (or some other fixed location). We will use zero.
   • So when there is an internal jump in the program, say jump to location 100, this means jump to location 100 of the current module.
   • To convert this relative address to an absolute address, the linker adds the base address of the module to the relative address. The base address is the address at which this module will be loaded.
   • Example: Module A is to be loaded starting at location 2300 and contains the instruction
     jump 120
     The linker changes this instruction to
     jump 2420
   • How does the linker know that Module M5 is to be loaded starting at location 2300?
   • It processes the modules one at a time. The first module is to be loaded at location zero. So the relocating is trivial (adding zero). We say the relocation constant is zero.
   • When finished with the first module (say M1), the linker knows the length of M1 (say that length is L1).
   • Hence the next module is to be loaded starting at L1, i.e., the relocation constant is L1.
   • In general the linker keeps track of the sum of the lengths of all the modules it has already processed and this is the location at which the next module is to be loaded.
   
   
2. Resolving external references.

   

   • If a C (or Java, or Pascal) program contains a function call
     f(x)
     to a function f() that is compiled separately, the resulting object module will contain some kind of jump to the beginning of f.
   • But this is impossible!
   • When the C program is compiled. the compiler (and assembler) do not know the location of f() so there is no way it can supply the starting address.
   • Instead a dummy address is supplied and a notation made that this address needs to be filled in with the location of f(). This is called a use of f.
   • The object module containing the definition of f() indicates that f is being defined and gives its relative address (which the linker will convert to an absolute address). This is called a definition of f.
   
   

The output of a linker is called a load module because it is now ready to be loaded and run.

To see how a linker works lets consider the following example, which is the first dataset from lab #1. The description in lab1 is more detailed.

The target machine is word addressable and has a memory of 1000 words, each consisting of 4 decimal digits. The first (leftmost) digit is the opcode and the remaining three digits form an address.

Each object module contains three parts, a definition list, a use list, and the program text itself. Each definition is a pair (sym, loc). Each use is a pair (sym, loc). The address in loc points to the next use or is 999 to end the chain.

For those text entries that do not form part of a use chain a fifth (leftmost) digit is added. If it is 8, the address in the word is relocatable. If it is 0 (and hence omitted), the address is absolute.

Sample input
1 xy 2
1 z 4
5 81234 5678 2999 88888 7002
0
1 z 3
6 88888 1999 1001 3002 81002 1234
0
1 z 1
2 81234 4999
1 z 2
1 xy 2
3 8000 1999 2001

I will illustrate a two-pass approach: The first pass simply produces the symbol table giving the values for xy and z (2 and 15 respectively). The second pass does the real work (using the values in the symbol table).

It is faster (less I/O) to do a one pass approach, but is harder since you need ``fix-up code'' whenever a use occurs in a module that precedes the module with the definition.

xy=2
z=15

+0
0:      81234           1234+0=1234
1:       5678           5678
2: xy:   2999   ->z     2015
3:      88888           8888+0=8888
4: ->z   7002           7015
+5
0       88888           8888+5=8893
1        1999   ->z     1015
2        1001   ->z     1015
3 ->z    3002           3015
4       81002           1002+5=1007
5        1234           1234
+11
0       81234           1245
1 ->z    4999           4015
+13
0        8000           8000
1        1999   ->xy    1002    
2 z:->xy 2001           2002

The linker on unix is mistakenly called ld (for loader), which is unfortunate since it links but does not load.

Lab #1 Implement a linker. The specific assignment is detailed on the sheet handed out in in class and is due in three weeks 16 February. The content of the handout is available on the web as well (see the class home page).

End of Interlude on Linkers

Chapter 1. Introduction

Homework: Read Chapter 1 (Introduction)

Levels of abstraction (virtual machines)

• Software (and hardware, but that is not this course) is often implemented in layers.
• The higher layers use the facilities provided by lower layers.
• Alternatively said, the upper layers are written using a more powerful and more abstract virtual machine than the lower layers.
• Alternatively said, each layer is written as though it runs on the virtual machine supplied by the lower layer and in turn provides a more abstract (pleasent) virtual machine for the higher layer to run on.
• Using a broad brush, the layers are.
  1. Scripts (e.g. shell scripts)
  2. Applications and utilities
  3. Libraries
  4. The OS proper (the kernel)
  5. Hardware
• The kernel itself is itself normally layered, e.g.
  1. ...
  2. Filesystems
  3. Machine independent I/O
  4. Machine dependent device drivers
• The machine independent I/O part is written assuming ``virtual (i.e. idealized) hardware''. For example, the machine independent I/O portion simply reads a block from a ``disk''. But in reality one must deal with the specific disk controller.
• Often the machine independent part is more than one layer.
• The term OS is not well defined. Is it just the kernel? How about the libraries? The utilities? All these are certainly system software but not clear how much is part of the OS.

1.1: What is an operating system?

The kernel itself raises the level of abstraction and hides details. For example a user (of the kernel) can write to a file (a concept not present in hardware) and ignore whether the file resides on a floppy, a CD-ROM, or a hard magnetic disk

The kernel is a resource manager (so users don't conflict).

How is an OS fundamentally different from a compiler (say)?

Answer: Concurrency! Per Brinch Hansen in Operating Systems Principles (Prentice Hall, 1973) writes.

The main difficulty of multiprogramming is that concurrent activities can interact in a time-dependent manner, which makes it practically impossibly to locate programming errors by systematic testing. Perhaps, more than anything else, this explains the difficulty of making operating systems reliable.

1.2 History of Operating Systems

1. Single user (no OS).
2. Batch, uniprogrammed, run to completion.
   • The OS now must be protected from the user program so that it is capable of starting (and assisting) the next program in the batch).
3. Multiprogrammed
   • The purpose was to overlap CPU and I/O
   • Multiple batches
     • IBM OS/MFT (Multiprogramming with a Fixed number of Tasks)
       • The (real) memory is partitioned and a batch is assigned to a fixed partition.
       • The memory assigned to a partition does not change
     • IBM OS/MVT (Multiprogramming with a Variable number of Tasks) (then other names)
       • Each job gets just the amount of memory it needs. That is, the partitioning of memory changes as jobs enter and leave
       • MVT is a more ``efficient'' user of resources but is more difficult.
       • When we study memory management, we will see that with varying size partitions questions like compaction and ``holes'' arise.
   • Time sharing
     • This is multiprogramming with rapid switching between jobs (processes). Deciding when to switch and which process to switch to is called scheduling.
     • We will study scheduling when we do processor management
4. Multiple computers
   • Multiprocessors: Almost from the beginning of the computer age but now are not exotic.
   • Network OS: Make use of the multiple PCs/workstations on a LAN.
   • Distributed OS: A ``seamless'' version of above.
   • Not part of this course (but often in G22.2251).
5. Real time systems
   • Often in embedded systems
   • Soft vs hard real time. In the latter missing a deadline is a fatal error--sometimes literally.
   • Very important commercially, but not covered much in this course.

1.3: Operating System Concepts

This will be very brief. Much of the rest of the course will consist in ``filling in the details''.

1.3.1: Processes

A program in execution. If you run the same program twice, you have created two processes. For example if you have two editors running in two windows, each instance of the editor is a separate process.

Often one distinguishes the state or context (memory image, open files) from the thread of control. Then if one has many threads running in the same task, the result is a ``multithreaded processes''.

The OS keeps information about all processes in the process table. Indeed, the OS views the process as the entry. An example of an active entity being viewed as a data structure (cf. discrete event simulations). An observation made by Finkel in his (out of print) OS textbook.

The set of processes forms a tree via the fork system call. The forker is the parent of the forkee. If the parent stops running until the child finishes, the ``tree'' is quite simple, just a line. But the parent (in many OSes) is free to continue executing and in particular is free to fork again producing another child.

A process can send a signal to another process to cause the latter to execute a predefined function (the signal handler). This can be tricky to program since the programmer does not know when in his ``main'' program the signal handler will be invoked.

1.3.2: Files

Modern systems have a hierarchy of files. A file system tree.

• In MSDOS the hierarchy is a forest not a tree. There is no file, or directory that is an ancestor if both a:\ and c:\.
• In unix the existence of symbolic links weakens the tree to a DAG

Files and directories normally have permissions

• Normally have at least rwx.
• User, group, world
• More general is access control lists
• Often files have ``attributes'' as well. For example the linux ext2 filesystem supports a ``d'' attribute that is a hint to the dump program not to backup this file.
• When a file is opened, permissions are checked and, if the open is permitted, a file descriptor is returned that is used for subsequent operations

Devices (mouse, tape drive, cdrom) are often view as ``special files''. In a unix system these are normally found in the /dev directory. Often utilities that are normally applied to (ordinary) files can be applied as well to some special files. For example, when you are accessing a unix system and do not have anything serious going on (e.g., right after you log in), type the following command

    cat /dev/mouse

Many systems have standard files that are automatically made available to a process upon startup. There (initial) file descriptors are fixed

• standard input: fd=0
• standard output: fd=1
• standard error: fd=2

A convenience offered by some command interpretors is a pipe or pipeline. The pipeline

  ls | wc

Homework: 3