Class 4 CS 202 01 February 2023 On the board ------------ 1. Last time 2. The shell 3. File descriptors 4. The shell, part II 5. Processes: the OS's view 6. Threads 7. Intro to concurrency -------------------------------------------------------------------------- 1. Last time - Stack frames, continued - System calls - Process/OS control transfers - Git/lab setup - Process birth - Shell pipeline demo 2. The shell, part I --a program that creates processes --the human's interface to the computer --GUIs (graphical user interfaces) are another kind of shell. A. How does the shell start programs? --example: $ ls [see panel 1 on handout; go line-by-line] --calls fork(), which creates a copy of the shell. now there are two copies of the shell running --then calls exec(), which loads the new program's instructions into memory and begins executing them. --(exec invokes the loader) while (1) { write(1, "$ ", 2); readcommand(command, args); // parse input if ((pid = fork()) == 0) // child? execve(command, args, 0); else if (pid > 0) // parent? wait(0); //wait for child else perror("failed to fork"); } --how can shell wait for the end of a process? --with wait() or waitpid() system calls [show commandline interface] --background process. example: $ sleep 10 vs $ sleep 10 & --QUESTION: why is fork different from exec? What the ...? * We will come back to this. B. Redirection, motivation What does this do? $ ./first3 abcd efgh > foo What about this? $ ps xc | grep ... And remember the pipeline demo from last time $ cat blob | grep -o labs-22sp-[a-zA-Z0-9\-]* | sort -f | uniq > students.txt How are these things implemented? Remember, the programmer of first3 or cat or grep is long gone, and their output is winding up somewhere that the original program never specified. 3. File descriptors --every process can usually expect to begin life with three file descriptors already open: 0: represents the input to the process (e.g., tied to terminal) 1: represents the output 2: represents the error output these are sometimes known as stdin, stdout, stderr --NOTE: Unix hides for processes the difference between a device and a file. this is a very powerful hiding (or abstraction), as we will see soon 4. The shell, part II - Back to $ ./first3 abcd efgh > /tmp/foo How is that implemented? Answer: just before exec, shell does: close(1) open("/tmp/foo", O_TRUNC | O_CREAT | O_WRONLY, 0666) which automatically assigns fd 1 to point to /tmp/foo. --now, when first3 runs, it still has in its code: write(1,...), but "1" now means something else. What about $ sh < script > tmp1 where script contains echo abc echo def [draw picture] - Pipelines [for now, leaving this as an exercise] - The power of the fork/exec separation [an innovation from the original Unix. possibly lucky design choice at the time. but turns out to work really well. allows the child to manipulate environment and file descriptors *before* exec, so that the *new* program may in fact encounter a different environment] --To generalize redirections and pipelines, there are lots of things the parent shell might want to manipulate in the child process: file descriptors, environment, resource limits. --yet fork() requires no arguments! --Contrast with CreateProcess on Windows: BOOL CreateProcess( name, commandline, security_attr, thr_security_attr, inheritance?, other flags, new_env, curr_dir_name, .....) [http://msdn.microsoft.com/en-us/library/ms682425(v=VS.85).aspx] there's also CreateProcessAsUser, CreateProcessWithLogonW, CreateProcessWithTokenW, ... * The issue is that any conceivable manipulation of the environment of the new process has to be passed through arguments, instead of via arbitrary code. in other words: because whoever calls CreateProcess() (or its variant) needs to perfectly configure the process before it starts running. with fork(), whoever calls fork() **is still running** so can arrange to do whatever it wants, without having to work through a rigid interface like the above. allows arbitrary "setup" of the process before exec(). - Discussion: what makes a good abstraction? --simple but powerful --examples we've seen: --stdin (0), stdout (1), stderr (2) [nice by themselves, but when combined with the mechanisms below, things get even better] --file descriptors --fork/exec() separation --very few mechanisms lead to a lot of possible functionality 5. Implementation of processes Briefly cover the OS's view: PCB ----------------- | process id | | state | (ready, runnable, blocked, etc.) | user id | | IP (ins ptr)| | open file | | VM structures | | registers | | ..... | (signal mask, terminal, priority, ...) ---------------- called "proc" in Unix, "task_struct" in Linux, and "process_t" in lab4. [draw an array of these.] point out that during scheduling, a mechanism that we have not seen, a core switches between processes. will discuss the mechanism for this later. Note: these PCBs will have an analog when considering threads, below. 6. Threads Interface to threads: tid thread_create (void (*fn) (void *), void *); Create a new thread, run fn with arg void thread_exit (); void thread_join (tid thr); Wait for thread with tid 'thr' to exit plus a lot of synchronization primitives, which we'll see in the coming classes Assume for now that threads are: --an abstraction created by OS --preemptively scheduled [draw abstract picture of threads: own registers, share memory] (later, we will explore alternatives) 7. Intro to concurrency There are many sources of concurrency. --what is concurrency? --stuff happening at the same time --sources of concurrency --computers have multiple CPUs and common memory, so instructions in multiple threads can happen at the same time! --on a single CPU, processes/threads can have their instructions interleaved (helpful to regard the instructions in multiple threads as "happening at the same time") --interrupts (CPU was doing one thing; now it's doing another) --why is concurrency hard? *** Hard to reason about all possible interleavings --handout: 1a: x = 1 or x = 2. 1b: x = 13 or x = 25. 1c: x = 1 or x = 2 or x = 3 say x is at mem location 0x5000 f is "x = x+1;", which might compile to: movq 0x5000, %rbx # load from address 0x5000 into register addq $1, %rbx # add 1 to the register's value movq %rbx, 0x5000 # store back g is "x = x+2;", which might compile to: movq 0x5000, %rbx # load from address 0x5000 into register addq $2, %rbx # add 2 to the register's value movq %rbx, 0x5000 # store back 2: incorrect list structure 3: incorrect count in buffer all of these are called race conditions; not all of them are errors, though --worst part of errors from race conditions is that a program may work fine most of the time but only occasionally show problems. why? (because the instructions of the various threads or processes or whatevever get interleaved in a non-deterministic order.) --and it's worse than that because inserting debugging code may change the timing so that the bug doesn't show up