Class 22 CS 439 04 April 2013 --------------- [lecturer: Navid Yaghmazadeh] * agenda: - Client/Server systems - RPC (Remote Procedure Call) - NFS (Network File system) -- Intro/motivation -- Architecture -- Mechanics ################################################### 1. Client/Server Systems: -- Up to now what we have seen run on one machine [draw first one machine] -- Processes running on multiple machines (web browsing) [draw second machine] -- need to communicate (functionalities on different mahines) -- network abstraction for communication -- How to communicate? ################################################## 2. RPC (Remote Procedure Call) -- Calling function on other machine (compare to local function call) -- Inter-process communication (vs. shared memory, message passing, …) -- How it works? [draw pic1] -- A sends a message to B, invokes some functionality on B -- client/server stub (RPC libraries, OS, …) -- potential of RPC: fantastic way to build distributed systems -- RPC system takes care of all the distributed/network issues (packet drops, network down, …) -- factoring out security, synchronization, and data flow -- Idea is everywhere -- Most of distributed systems/applications (https[post, get,..], 2PC, ..) -- Great for client/server systems -- Example? where we used it? ###################################################### 3. NFS 3.1. Motivation: -- UTCS machines -- Create/modify some file when you logged in machine A -- Read that file in future from machine B -- What's the magic here? Does the file fly? -- Networked File System: -- What's a network file system? -- Looks like a file system (e.g., FFS) to applications -- But data potentially stored on another machine -- Reads and writes must go over the network -- Also called distributed file systems -- Advantages of network file systems -- Easy to share if files available on multiple machines -- Often easier to administer servers than clients -- Access way more data than fits on your local disk -- Network + remote buffer cache faster than local disk --Disadvantages -- Network + remote disk slower than local disk -- Network or server may fail even when client OK -- Complexity, security issues -- NFS: case study of client/server, case study of RPC, and case study of network file system 3.2. Introduction and background: -- Reasons to study it -- case study of RPC transparency -- NFS was very successful. -- Still in widespread use today (we're using it in class machines). -- Much research uses it. -- Can view much networked file systems research as fixing problems with NFS -- Background and context -- Designed in mid 1980s -- Before this, Sun was selling Unix workstations -- Diskless (to save money) -- "ND" network disk protocol (use one big central disk, and let the diskless workstations use it) -- Allowed disk to live somewhere else, but did not allow for shared file system (every workstation had a partitioned piece of the ND *not* a shared file system) --Advantages: -- Convenience (get your files anywhere) -- Cost (buy workstations without disks) 3.3. How it works: -- What's the software/hardware structure? [draw pic2] -- vnode like a primitive C++ or Java object, with methods --represents an open (or openable) file -- Bunch of generic "vnode operations": --lookup, create, open, close, getattr, setattr, read, write, fsync, remove, link, rename, mkdir, rmdir, symlink, readdir, readlink, ... -- Called through function pointers, so most system calls don't care what type of file system a file resides on -- We have seen this function-pointer-as-abstraction pattern before with device drivers (more generally, it shows up everywhere). -- NFS implements vnode operations through RPC -- Client request to server over network, awaits response -- Each system call may require a series of RPCs -- System mostly determined by NFS RPC **protocol** --How does it work? [TRACE RPC FOR OPEN AND WRITE: LOOKUP AND WRITE] [draw pic 3] -- Mounted /users/navid on NFS (on NFS we have a partition /navid) -- NFS client has dirfh for /users -- Nice separation between interface and implementation --loopback server --replace NFS server altogether with something that *acts* like an NFS server to the client. --can make lots of things *look* like a file system just by implementing the NFS interface. extremely powerful technique --this gain mostly arises because of the power of RPC and modularity, rather than anything about NFS in particular -- What does a file handle look like? [FS ID | inode # | generation #] -- Why not embed file name in file handle? (file names can change; would mess everything up. client needs to use an identifier that's invariant across such renames.) --what is generation number for? (*) Clients A and B both reading the same file (same inode), then client A deletes that file, and creates a new file which uses the same i-node. What happens if client B tries to access the file using the i-node it has? -- generation number prevents --Stale FH error More detail: For *all* files that could ever be exposed by NFS, the server stores, in the i-node on disk, a generation number. Every time the server allocates a given i-node, it increments the i-node's generation number. When the server passes a FH to the client (say, in response to a LOOKUP RPC from the client), the server puts the given i-node's _current_ generation number in the FH. How: The way the generation number avoids problems that arise from the special case in (*) is as follows: for each request the client makes of the server, the server checks to see whether the generation number in the client's FH matches the on-disk generation number for the i-node in question. If so, the client has a current FH, and the special case has not arisen. If not, the client's generation number must be older, so we are in the special case, and the client gets a "stale FH" error when it tries to READ() or WRITE(). Why: Without the generation number, the special case in (*) would cause a client to read and write data it had no business reading or writing (since the given i-node now belongs to some other file). -- How does client know what file handle to send? (stored with the vnode) 3.4. Statelessness: -- What the heck do they mean? The file server keeps files; that's certainly state!! -- What they really mean is that every network protocol request contains all of the information needed to carry out that request, without relying on anything remembered from previous protocol requests. -- But are operations really idempotent? -- what happens if two renames() are sent, and the reply to the first one is lost? client sends another one. then the second one returns an error code, even though the operation conceptually succeeded. -- What are the advantages and disadvantages? +: simplifies implementation +: simplifies server failure recovery -: messes up traditional Unix semantics; will discuss below --NOTE: a crashed and rebooted server looks the same to clients as a slow server. Which is cool. detour [consistency vs performance]: --Server must flush to disk before returning (why?) --Inode with new block # and new length safe on disk. --Indirect block safe on disk. --So writes have to be synchronous --so why isn't performance bad? caching. not all RPCs actually go to server [see (iv) below] 3.5. Transparency and non-traditional Unix semantics: -- Note: transparency is not just about preserving the syscall API (which they do). Transparency requires that the system calls *mean* the same things. Otherwise, existing programs may compile and run but experience different behavior. In other words, formerly correct programs may now be incorrect. (This happened with NFS because of its close-to-open consistency.) -- Remember why generation number was necessary. -- served file systems must support -- So not fully transparent -- non-traditional Unix semantics (i) we mentioned one example above: error returns on successful operations. now we'll go through some other examples of changed semantics (ii) server failure --previously, open() failed only if file didn't exist --now, if server has failed, open() can fail or apps hang [fundamental trade-off if server is remote] (iii) deletion of open files --What if client A deletes a file that client B has "open"? --Unix: my reads still work (file exists until all clients close() it) --NFS: my reads fail --Why? --To get Unix-like behavior using NFS, server would have to keep track of all kinds of stuff. That state would have to persist across reboots. --But they wanted stateless server --So NFS just does the wrong thing --RPCs fail if another client deletes a file you have open. --(Hack if the *same* client does the delete: the NFS client asks the NFS server to do a rename. that's where stale file handles come from. more on this below.) (iv) chmod -r while file is open() (in Unix, nothing happens. in NFS, future reads fail, since NFS checks permissions on every RPC.) (v)execute-only implies read, unlike in Unix (in Unix, the operating system draws a distinction between demand-paging in the executable, to execute it, versus returning bytes in a file to a requesting program. a user might have permission to do one, or the other, or both. in NFS, the NFS server cannot care about this distinction because the NFS client needs the data blocks in the file, period. thus, if a file is marked execute-only on the NFS server, the NFS client will still be able to read it if the NFS client really wants (once the NFS client has the data blocks, it has the data blocks). 3.6. Security --Only security is via IP address --Another case of non-transparency: --On local system: UNIX enforces read/write protections Can't read my files w/o my password --On NFS: --Server believes whatever UID appears in NFS request --Anyone on the Internet can put whatever they like in the request --Or you (on your workstation) can su to root, then su to me --2nd su requires no password --Then NFS will let you read/write my files --In other words, to steal data, just adopt the uid of the person whose files you're trying to read....or just spoof packets. --So why aren't NFS servers ridiculously vulnerable? --Hard to guess correct file handles. --(Which rules out one class of attacks but not spoofed UIDs) --Observe: the vulnerabilities are fixable --Other file systems do it --Require clients to authenticate themselves cryptographically. --But very hard to reconcile with statelessness. 3.7. Concluding note --None of the above issues prevent NFS from being useful. --People fix their programs to handle new semantics. --Or install firewalls for security. --And get most advantages of transparent client/server. [This note is a subversion of Mike Walfish's note on RPC and NFS, which is partially based on notes by Mike Dahlin and Robert Morris.]