Operating Systems

Start Lecture #4

2.2.2 The Classical Thread Model

A process contains a number of resources such as address space, open files, accounting information, etc. In addition to these resources, a process has a thread of control, e.g., program counter, register contents, stack. The idea of threads is to permit multiple threads of control to execute within one process. This is often called multithreading and threads are sometimes called lightweight processes. Because threads in the same process share so much state, switching between them is much less expensive than switching between separate processes.

Individual threads within the same process are not completely independent. For example there is no memory protection between them. This is typically not a security problem as the threads are cooperating and all are from the same user (indeed the same process). However, the shared resources do make debugging harder. For example one thread can easily overwrite data needed by another thread in the process and when the second thread fails, the cause may be hard to determine because the tendency is to assume that the failed thread caused the failure.

A new thread in the same process is created by a routine named something like thread_create; similarly there is thread_exit. The analogue to waitpid is thread_join (the name comes presumably from the fork-join model of parallel execution).

The routine tread_yield, which relinquishes the processor, does not have a direct analogue for processes. The corresponding system call (if it existed) would move the process from running to ready.

Homework: 11.

Challenges and Questions

Assume a process has several threads. What should we do if one of these threads

1. Executes a fork?
2. Closes a file?
3. Requests more memory?
4. Moves a file pointer via lseek?

2.2.3 POSIX Threads

POSIX threads (pthreads) is an IEEE standard specification that is supported by many Unix and Unix-like systems. Pthreads follows the classical thread model above and specifies routines such as pthread_create, pthread_yield, etc.

An alternative to the classical model are the so-called Linux threads (see the section 10.3 in the 3e).

2.2.4 Implementing Threads in User Space

Write a (threads) library that acts as a mini-scheduler and implements thread_create, thread_exit, thread_wait, thread_yield, etc. This library acts as a run-time system for the threads in this process. The central data structure maintained and used by this library is a thread table, the analogue of the process table in the operating system itself.

There is a thread table and an instance of the threads library in each multithreaded process.

Advantages of User-Mode Threads:

• Requires no OS modification.
• Requires NO OS modification.
• Requires NO OS modification.
• Very fast since no context switching.
• Can customize the scheduler for each application.

Disadvantages

• Blocking system calls can't be executed directly since that would block the entire process. For example, consider the producer consumer example above implemented in the natural manner with user-mode threads. This implementation would not work well since, whenever an I/O was issued that caused the process to block, all the threads would be unable to run (but see just below).
• Similarly a page fault would block the entire process (i.e., all the threads).
• in addition, a thread with an infinite loop prevents all other threads in this process from running.

Possible Methods of Dealing With Blocking System Calls

• Perhaps the OS supplies a non-blocking version of the system call, e.g. a non-blocking read.
• Perhaps the OS supplies another system call that tells if the blocking system call will in fact block. For example, a unix select() can be used to tell if a read would block. It might not block if, for example,
  • The requested disk block is in the buffer cache (see the I/O chapter).
  • The request was for a keyboard or mouse or network event that has already happened.

2.2.4 Implementing Threads in the Kernel

One can move the thread operations into the operating system itself. This naturally requires that the operating system itself be (significantly) modified and is thus not a trivial undertaking.

• There is only one thread table for the entire system and it is in the OS.
• Thread-create and friends are now system calls and hence much slower than with user-mode threads. They are, however, still much faster than creating/switching/etc processes since there is so much shared state that does not need to be recreated.
• A thread that blocks causes no particular problem. The kernel can run another thread from this process (or can run a thread from another process).
• Similarly a page fault or infinite loop in one thread does not automatically block the other threads in the process.

2.2.5 Hybrid Implementations

One can write a (user-level) thread library even if the kernel also has threads. This is sometimes called the N:M model since N user-mode threads run on M kernel threads. In this scheme, the kernel threads cooperate to execute the user-level threads.

• Different kernel threads in the same process can have differing numbers of user threads assigned to them.
• Switching between user-level threads within one kernel thread is very fast (no context switch). It is essentially the same as in the case of pure user-mode threads.
• Switching between kernel threads of the same process requires a system call and is essentially the same as in the case of pure kernel-level threads.
• Since a blocking system call or page fault blocks only one kernel thread, the multi-threaded application as a whole can still run since user-level threads in other kernel-level threads of this process are still runnable.

An offshoot of the N:M terminology is that kernel-level threading (without user-level threading) is sometimes referred to as the 1:1 model since one can think of each thread as being a user level thread executed by a dedicated kernel-level thread.

Homework: 12, 14.

Remark: We shall do section 2.4 before section 2.3 for two reasons.

1. Sections 2.3 and 2.5 are closely related; having 2.4 in between seeks awkward to me.
2. Lab 2 uses material from 2.4 so I don't want to push 2.4 after 2.5.

2.4 Process Scheduling

Scheduling processes on the processor is often called processor scheduling or process scheduling or simply scheduling. As we shall see later in the course, a more descriptive name would be short-term, processor scheduling.

For now we are discussing the arcs connecting running↔ready in the diagram on the right showing the various states of a process. Medium term scheduling is discussed later as is disk-arm scheduling.

Naturally, the part of the OS responsible for (short-term, processor) scheduling is called the (short-term, processor) scheduler and the algorithm used is called the (short-term, processor) scheduling algorithm.

2.4.1 Introduction to Scheduling

Importance of Scheduling for Various Generations and Circumstances

Early computer systems were monoprogrammed and, as a result, scheduling was a non-issue.

For many current personal computers, which are definitely multiprogrammed, there is in fact very rarely more than one runnable process. As a result, scheduling is not critical.

For servers (or old mainframes), scheduling is indeed important and these are the systems you should think of.

Process Behavior

Processes alternate CPU bursts with I/O activity, as we shall see in lab2. The key distinguishing factor between compute-bound (aka CPU-bound) and I/O-bound jobs is the length of the CPU bursts.

The trend over the past decade or two has been for more and more jobs to become I/O-bound since the CPU rates have increased faster than the I/O rates.

When to Schedule

An obvious point, which is often forgotten (I don't think 3e mentions it) is that the scheduler cannot run when the OS is not running. In particular, for the uniprocessor systems we are considering, no scheduling can occur when a user process is running. (In the mulitprocessor situation, no scheduling can occur when all processors are running user jobs).

Again we refer to the state transition diagram above.

1. Process creation.
   The running process has issued a fork() system call and hence the OS runs; thus scheduling is possible. Scheduling is also desirable at this time since the scheduling algorithm might favor the new process.
2. Process termination.
   The exit() system call has again transferred control to the OS so scheduling is possible. Moreover, scheduling is necessary since the previously running process has terminated.
3. Process blocks.
   Same as termination.
4. Interrupt received.
   Since the OS takes control, scheduling is possible. When an I/O interrupt occurs, this normally means that a blocked process is now ready and, with a new candidate for running, scheduling is desirable.
5. Clock interrupts are treated next when we discuss preemption and discuss the dotted arc in the process state diagram.

Preemption

It is important to distinguish preemptive from non-preemptive scheduling algorithms.

• Preemption means the operating system moves a process from running to ready without the process requesting it.
• Without preemption, the system implements run until completion, or block (, or yield).
• The preempt arc in the diagram is present for preemptive scheduling algorithms.
• We do not emphasize yield (a solid arrow from running to ready).
• Preemption needs a clock interrupt (or equivalent).
• Preemption is needed to guarantee fairness.
• Preemption is found in all modern general purpose operating systems.
• Even non-preemptive systems can be multiprogrammed (remember that processes do block for I/O).
• Preemption is not cheap.

Categories of Scheduling Algorithms

We distinguish three categories of scheduling algorithms with regard to the importance of preemption.

1. Batch.
2. Interactive.
3. Real Time.

For multiprogramed batch systems (we don't consider uniprogrammed systems, which don't need schedulers) the primary concern is efficiency. Since no user is waiting at a terminal, preemption is not crucial and if it is used, each process is given a long time period before being preempted.

For interactive systems (and multiuser servers), preemption is crucial for fairness and rapid response time to short requests.

We don't study real time systems in this course, but can say that preemption is typically not important since all the processes are cooperating and are programmed to do their task in a prescribed time window.

Scheduling Algorithm Goals

There are numerous objectives, several of which conflict, that a scheduler tries to achieve. These include.

1. Fairness.
   Treating users fairly, which must be balanced against ...
   
   
2. Respecting priority.
   That is, giving more important processes higher priority. For example, if my laptop is trying to fold proteins in the background, I don't want that activity to appreciably slow down my compiles and especially don't want it to make my system seem sluggish when I am modifying these class notes. In general, interactive jobs should have higher priority.
   
   
3. Efficiency.
   This has two aspects.
   • Do not spend excessive time in the scheduler.
   • Try to keep all parts of the system busy.
   
   
4. Low turnaround time
   That is, minimize the time from the submission of a job to its termination. This is important for batch jobs.
   
   
5. High throughput.
   That is, maximize the number of jobs completed per day. Not quite the same as minimizing the (average) turnaround time as we shall see when we discuss shortest job first.
   
   
6. Low response time.
   That is, minimize the time from when an interactive user issues a command to when the response is given. This is very important for interactive jobs.
   
   
7. Repeatability. Dartmouth (DTSS) wasted cycles and limited logins for repeatability.
   
   
8. Degrade gracefully under load.

The Name Game

There is an amazing inconsistency in naming the different (short-term) scheduling algorithms. Over the years I have used primarily 4 books: In chronological order they are Finkel, Deitel, Silberschatz, and Tanenbaum. The table just below illustrates the name game for these four books. After the table we discuss several scheduling policy in some detail.

    Finkel  Deitel  Silbershatz Tanenbaum
    -------------------------------------
    FCFS    FIFO    FCFS        FCFS
    RR      RR      RR          RR
    PS      **      PS          PS
    SRR     **      SRR         **    not in tanenbaum
    SPN     SJF     SJF         SJF
    PSPN    SRT     PSJF/SRTF   SRTF
    HPRN    HRN     **          **    not in tanenbaum
    **      **      MLQ         **    only in silbershatz
    FB      MLFQ    MLFQ        MQ
  

Remark: For an alternate organization of the scheduling algorithms (due to my former PhD student Eric Freudenthal and presented by him Fall 2002) click here.

2.4.2 Scheduling in Batch Systems

First Come First Served (FCFS, FIFO, FCFS, --)

If the OS doesn't schedule, it still needs to store the list of ready processes in some manner. If it is a queue you get FCFS. If it is a stack (strange), you get LCFS. Perhaps you could get some sort of random policy as well.

• Only FCFS is considered.
• Non-preemptive.
• The simplist scheduling policy.
• In some sense the fairest since it is first come first served. But perhaps that is not so fair. Consider a 1 hour job submitted one second before a 3 second job.
• An efficient usage of cpu in the sense that the scheduler is very fast.
• Does not favor interactive jobs.

Shortest Job First (SPN, SJF, SJF, SJF)

Sort jobs by execution time needed and run the shortest first.

This is a Non-preemptive algorithm.

First consider a static situation where all jobs are available in the beginning and we know how long each one takes to run. For simplicity lets consider run-to-completion, also called uniprogrammed (i.e., we don't even switch to another process on I/O). In this situation, uniprogrammed SJF has the shortest average waiting time.

• Assume you have a schedule with a long job right before a short job.
• Consider swapping the two jobs.
• This decreases the wait for the short by the length of the long job and increases the wait of the long job by the length of the short job.
• This decreases the total waiting time for these two.
• Hence decreases the total waiting for all jobs and hence decreases the average waiting time as well.
• Hence, whenever a long job is right before a short job, we can swap them and decrease the average waiting time.
• Thus the lowest average waiting time occurs when there are no short jobs right before long jobs.
• This is uniprogrammed SJF.

The above argument illustrates an advantage of favoring short jobs (e.g., RR with small quantum): The average waiting time is reduced.

In the more realistic case of true SJF where the scheduler switches to a new process when the currently running process blocks (say for I/O), we could also consider the policy shortest next-CPU-burst first.

The difficulty is predicting the future (i.e., knowing in advance the time required for the job or the job's next-CPU-burst).

SJF Can starve a process that requires a long burst.

• This is fixed by the standard technique.
• What is that technique?
  Answer: Priority aging (see below).

Shortest Remaining Time Next (PSPN, SRT, PSJF/SRTF, SRTF)

Preemptive version of above. Indeed some authors call it preemptive shortest job first.

• Permit a process that enters the ready list to preempt the running process if the time for the new process (or for its next burst) is less than the remaining time for the running process (or for its current burst).
• It will never happen that a process already in the ready list will require less time than the remaining time for the currently running process. Why?
  Answer: When the process joined the ready list it would have started running if the current process had more time remaining. Since that didn't happen the currently running job had less time remaining and now it has even less.
• PSPN Can starve a process that requires a long burst.
  • This is fixed by the standard technique.
  • What is that technique?
    Answer: Priority aging (see below).

2.4.3 Scheduling in Interactive Systems

The following algorithms can also be used for batch systems, but in that case, the gain may not justify the extra complexity.

Round Robbin (RR, RR, RR, RR)

• An important preemptive policy.
• Essentially the preemptive version of FCFS.
• The key parameter is the quantum size q.
• When a process is put into the running state a timer is set to q.
• If the timer goes off and the process is still running, the OS preempts the process.
  • This process is moved to the ready state (the preempt arc in the diagram), where it is placed at the rear of the ready list.
  • The process at the front of the ready list is removed from the ready list and run (i.e., moves to state running).
  • Note that the ready list is being treated as a queue. Indeed it is sometimes called the ready queue, but not by me since for other scheduling algorithms it is not accessed in a FIFO manner.
• When a process is created, it is placed at the rear of the ready list.
• Note that RR works well if you have a 1 hr job and then a 3 second job.
• As q gets large, RR approaches FCFS. Indeed if q is larger that the longest time any process will run before terminating or blocking, then RR IS FCFS. A good way to see this is to look at my favorite diagram and note the three arcs leaving running. They are triggered by three conditions: process terminating, process blocking, and process preempted. If the first trigger condition to arise is never preemption, we can erase that arc and then RR becomes FCFS.
• As q gets small, RR approaches PS (Processor Sharing, described next).
• What value of q should we choose?
  • A trade-off
  • A small q makes the system more responsive, a long compute-bound job cannot starve a short job.
  • A large q makes the system more efficient since there is less process switching.
  • A reasonable time for q is a few tens of milliseconds or perhaps a few milliseconds for a fast processor. (millisecond = 1/1000 second and is abbreviated ms). This means every other job can delay your job by at most q (plus the context switch time CS, which is normally less than 1ms). Also the overhead is CS/(CS+q), which is small.
• A student found the following reference for the name Round Robin. A similar, but less detailed, citation can be found in wikipedia.

  The round robin was originally a petition, its signatures arranged in a circular form to disguise the order of signing. Most probably it takes its name from the ruban rond, (round ribbon), in 17th-century France, where government officials devised a method of signing their petitions of grievances on ribbons that were attached to the documents in a circular form. In that way no signer could be accused of signing the document first and risk having his head chopped off for instigating trouble. Ruban rond later became round robin in English and the custom continued in the British navy, where petitions of grievances were signed as if the signatures were spokes of a wheel radiating from its hub. Today round robin usually means a sports tournament where all of the contestants play each other at least once and losing a match doesn't result in immediate elimination.
  
  Encyclopedia of Word and Phrase Origins by Robert Hendrickson (Facts on File, New York, 1997).

Homework: 20, 35.

Homework: Round-robin schedulers normally maintain a list of all runnable processes, with each process occurring exactly once in the list. What would happen if a process occurred more than once in the list? Can you think of any reason for allowing this?

Homework: Give an argument favoring a large quantum; give an argument favoring a small quantum.

Process	CPU Time	Creation Time
P1	20	0
P2	3	3
P3	2	5

Homework:

• Consider the set of processes in the table to the right.
• When does each process finish if RR scheduling is used with q=1, if q=2, if q=3, if q=100?
• First assume (unrealistically) that context switch time is zero.
• Then assume it is .1. Each process performs no I/O (i.e., no process ever blocks).
• All times are in milliseconds.
• The CPU time is the total time required for the process (excluding any context switch time).
• The creation time is the time when the process is created. So P1 is created when the problem begins and P3 is created 5 milliseconds later.
• If two processes have equal priority (in RR this means if they both enter the ready state at the same cycle), we give priority (in RR this means place first on the queue) to the process with the earliest creation time. If they also have the same creation time, then we give priority to the process with the lower number.
• Remind me to discuss this last one in class next time.

Homework: Redo the previous homework for q=2 with the following changes. After process P1 runs for 3ms (milliseconds), it blocks for 2ms. P1 never blocks again. P2 never blocks. After P3 runs for 1 ms it blocks for 1ms. Assume the context switch time is zero. Remind me to answer this one in class next lecture.

Processor Sharing (PS, **, PS, PS)

Merge the ready and running states and permit all ready jobs to be run at once. However, the processor slows down so that when n jobs are running at once, each progresses at a speed 1/n as fast as it would if it were running alone.

• Clearly impossible as stated due to the overhead of process switching.
• Of theoretical interest (easy to analyze).
• Approximated by RR when the quantum is small. Make sure you understand this last point. For example, consider the last homework assignment (with zero context switch time and no blocking) and consider q=1, q=.1, q=.01, etc.
• Show what happens for 3 processes, A, B, C, each requiring 3 seconds of CPU time. A starts at time 0, B at 1 second, C at 2.
• Consider three processes all starting at time 0. One requires 1ms, the second 100ms, the third 10sec (seconds). Compute the total/average waiting time for RR q=1ms, PS, SJF, SRTN, and FCFS. Note that this depends on the order the processes happen to be processed in. The effect is huge for FCFS, modest for RR with modest quantum, and non-existent for PS and SRTN.

Homework: 32.