Class 1 CS 202 28 January 2020 On the board ------------ CS202(-003): Operating Systems Instructor: Michael Walfish TAs: Ioanna T., Sahana U. http://www.cs.nyu.edu/~mwalfish/classes/20sp 1. Introduction and goals 2. What is an operating system? 3. This course: Why and how will we study OSes 4. This course: Course mechanics 5. History 6. Processes --------------------------------------------------------------------------- 1. Introduction and goals --Hello --Introduce TAs --Class goals a. learn how systems, especially operating systems, work b. learn abstractions and concepts in operating systems... c. ... which will be useful beyond OSes (other large-scale systems, etc.) 2. What is an operating system? [draw picture of hardware (memory, CPU, disk, I/O), OS, user-level programs] [OS includes OS services: processes, virtual memory, file system (file contents, directories and file names), security, networking, inter-process communication, time, terminals, etc.] --Purpose of OS: provide services to user-level programs --Definition: An operating system creates, for processes, a machine that is easier to program than the raw hardware would be. --this software is classically described as doing two things: 1. managing the resources of the machine --example: scheduling -- give every process some of the CPU --example: virtual memory: give every process some physical memory resource management means that one bad program doesn't screw up another. OS does: --multiplexing --isolation, protection --sharing 2. abstracting the hardware --hide details of hardware (OS gets dirty; programmers don't) --hardware is nasty to program directly! --consider what is involved in getting things to the disk..... this provides essential convenience and portability. you really don't want applications to have to program the bare hardware --would lead to lots of repeated code --and unclear how to run more than one application at once, particularly more than one that are mutually distrustful. other modern relevance: such abstraction also allows the operating system to make changes to the underlying hardware without impacting applications. For example, as you walk around browsing the internet on a phone, your phone is switching between WiFi and cellular networks. By abstracting away details of the network, applications are entirely unaware of this switch and it appears seamless. Similarly, laptops will switch between two GPUs in order to save power when using a battery. Abstracting the hardware ensures that applications do not need to be modified when adopting such techniques. --what are examples of managing resources and providing abstractions? file systems, memory, I/O, scheduling * file systems: --abstraction: illusion file is continuous array of bytes; it's not fd = open("/tmp/foo", WR_ONLY) rc = write(fd, "abc...z", 26) abstractions here: --files --file descriptors --location of the file: applications can be entirely unaware of whether the file they are writing to goes on a hard disk inside the computer, a USB stick attached to the computer, a remote storage service like Dropbox or iCloud, etc. --isolation: user program can't write to a file unless it has permission. in some cases, for example, smart phones, the system goes even further and gives each application the impression that it is the only one who can change the file system. This has become particularly important in order to ensure that applications cannot stomp on each other's data, and reduce the kinds of errors the programmer must consider when writing programs. * text input: -- abstraction: illusion that input, whether coming from a soft keyboard displayed on a touch screen, a physical keyboard, or things like Braille keyboard all act the same. Programs are unaware of your input method. (Ultimately looks like a file.) -- isolation: need to ensure that keystrokes go to a single program. Particularly important when entering sensitive inputs such as passwords. Modern operating systems provide isolated text entry paths just to avoid leaks for this input. * memory: movl 0x1248, %edx [means "get contents of 0x1248 and put in %edx"] --abstraction: user program thinks it is reading from 0x1248; it is not --more generally, user program thinks it has a linear, contiguous address space; it does not --isolation: user program can't write to another user's memory * scheduling: --abstraction: process has the illusion that it is running continuously; it is not --isolation: user program that is hogging CPU gets switched out in favor of another user's program ***these are different points*** --you could imagine telling the user program that it was getting switched out 3. This course: Why and how will we study OSes? (i) Why? a. ideas are everywhere: resource management and abstraction, for example. b. understand fundamental design trade-offs --between performance and simplicity --between putting functions in hardware or software --code must be efficient (so low-level?) ... --...but abstract/portable (so high-level?) --code must be powerful (so many features?) --...but simple (thus a few composable building blocks?) --another trade-off: between security and convenience --and there are difficult interactions: --among features --among behaviors: scheduler and memory allocator --etc. c. useful to know "how things work". peek under the hood. -- doesn't matter what types of programs you work on, knowing what is happening under the hood is essential to debugging and improving performance. If you are building phone applications, understanding what is going on in the OS helpful with improving performance and battery life. If you are using DNNs (deep neural networks), some of the concerns from OSes impact how you batch things, how you schedule, etc. d. at the same time, there are still unsolved problems! --OS security remains a problem area. people defend their operating systems by deploying firewalls. --multicore: yes, the operating systems run, but there's an argument that perhaps the abstractions should be different (interaction between caches and address space abstractions) e. implementation-wise: --the environment can be unforgiving (can have weird h/w, no debugger, ...) (ii) How? a. Learn how stuff works --sometimes through case studies b. Learn specific techniques --time-tested solutions to hard problems --avoid "hacks" --examples: concurrent programming, journaling c. Learn how to approach problems --fundamental issues --concept of a design space d. not a priority: details of deployed OSes (e.g., details of Linux), but we may sometimes use these as examples e. this class: lots of discussion of OS/HW boundary and process/OS boundary (interfaces). Less attention to detailed implementation of OS. 4. This course: mechanics and admin [write on the board] communication us-to-you: 3 ways: Web page: check this every day Piazza email you-to-us: Piazza for coursework staff email list for admin/sensitive things components of the course: --lectures --labs --exams --reading --homeworks grading policies --lectures: --please ask questions --will write on board; paces lecture --will put lecture notes online --no laptops --labs: --key piece of the course --one of our labs will be a miniOS that boots on real hardware --labs should actually be fun --often: not much code to write (but lots to learn!) --I used to tell students, "Start early". Now I say, "Start on time". --Regardless, you need to allocate time. --I'm expecting you to feel challenged by the labs. The concept of "no pain, no gain" applies to learning. --exams: ad-hoc quizzes midterm (3/12/20) final --reading: see course page --homeworks: see course page --grading: homework and quizzes: 5% lab assignments: 20% midterm: 25% final: 50% --policies: please see the Web page, and let me say here: --help on the labs: please ask for lab, but please make sure that you've really thought through your question on your own. --the collaboration and academic integrity policy is real what's happened in prior years... --sports analogy 5. Abridged history --We'll begin the story with Unix, but operating systems go back earlier. --Unix: 1969 to early 1970s: Unix (from Thompson and Ritchie at Bell Labs) --goal was to run on cheap hardware. PDP-7 --eventually got a PDP-11: only $40-$60k! --> $250k-$375k in 2019 dollars --small system: 9000 LOC --no protection: https://www.bell-labs.com/usr/dmr/www/odd.html --Eventually, Thompson and Ritchie decide they need a new programming language to write Unix in --So C gets born --Unix took over the world. --Its abstractions are still in use everywhere --(Which is arguably depressing) --great example of a small number of mechanisms going very far (high ratio of capabilities to mechanism) a. stuff was added to Unix in the 1980s, at Berkeley b. Andy Tanenbaum: "System 7 was a dramatic improvement over its predecessors, and over its successors as well". --Separate strand: in the 1970s, at Xerox PARC, people are trying to build the personal computer.... --eventually led to the graphical interfaces and idioms we have today (windowing systems, mouse, menus) --hard to imagine why "personal computer" was revolutionary, but keep in mind that when the PARC team started, telling someone that you were going to build a personal computer was like telling them that you would have a personal nuclear reactor (computers were totally slow when they started, but those designers knew that computers would be fast, so they started building the thing people want and worried later about performance) 6. Processes - key abstraction: process - motivation: you want your computer to be able to do multiple things at once: - you might find it annoying to write code without being able to do anything else during that time. - or if we have a computer with multiple users, they all need to get things done simultaneously - another motivation: resource efficiency: -- example #1: increase CPU utilization: --->|wait for input|---->|wait for input| gcc----------------> [this is called "overlapping I/O and computation"] -- example #2: reduce latency: A goes for 80 s, B goes for 20 s A-----------> B --> : takes B 100 s run A and B concurrently, say for 10s each, makes B finish faster. [DRAW PICTURE: loader HUMAN --> SOURCE CODE --> EXECUTABLE -----------> PROCESS vi gcc as ld loader HUMAN ---> foo.c ---> foo.s ----> foo.o ---> a.out ----> process NOTE: 'ld' is the name of the linker. it stands for 'linkage editor'. ] classical definition of a process: instance of a running program example: --browser, text editor, word processor, PDF viewer, image processor, photo editor, messaging app - a process can be understood in two ways: from the **process's** point of view next class. high-level: process sees an abstract machine (more accurately, an abstract machine plus lots of calls for manipulating and interacting with that abstract machine.) from the **OS's** point of view meaning how does the OS implement, or arrange to create, the abstraction of a process? we will deprioritize this for now, and come back later in the course.