Class 13 CS 480-008 22 March 2016 On the board ------------ 1. Last time 2. Bug finding and program correctness 3. Symbolic execution intro 4. EXE --------------------------------------------------------------------------- 1. Last time --capabilities, a kind of isolation mechanism today, we take detour. not about isolation....different strategy for defending against security vulnerabilities: try to get less buggy programs. 2. Bug finding and program correctness [all of this is the domain of PL (programming languages) and FM (formal methods). These overlap with systems in the context of work like EXE.] --the ideal: prove your program completely correct this can be done in principle, but in practice it is a lot of work for small and medium-sized programs not practical for big programs (but people are working on it. we might get there someday.) but if you do achieve it, then your program has no security vulnerabilities (provided the spec was right and provided the proving tools, and the platform they run-on, are also bug-free.) --prove your programs is free of important and large classes of errors this can be done for large programs. For example, the Astree system of NYU Prof. Patrick Cousot has been used to validate the software that runs on Airbus planes. Hundred of thousands of lines of code. --write asserts in your program (this is a good practice) idea: the programmer knows what a logic bug looks like, and writes statements, called asserts, that detect such a bug. abstractly speaking, such bugs are violations of complex invariants on internal state. for example: after a transfer between accounts, one would write an assert: assert(sum_of_new_balances == sum_of_old_balances) asserts are sometimes expensive and often turned off (compiled away) in deployed code. but production code usually has them in the source. so the question, during testing, is: is it possible for adversarial input to cause the code to fail the checks? how can we find out? --write a test suite (you should do this) each test runs the program with some input; the test knows the expected output; the asserts in the code then get a chance to catch errors advantages: + good for intended functionality + good for bugs that are already known disadvantages: - not so good for unintended functionality == vulnerabilities - not so good for as-yet-unknown bugs - hard to write tests that get complete "coverage" == cause every line of code to execute if no test causes some line of code to execute, you can't spot bugs in it - takes effort to write tests so, this raises questions: can we automate test-case generation? can we automate search for as-yet-unknown bugs? can we automate generation of tests that achieve good coverage? --fuzz testing idea: execute program on lots of randomly-generated inputs a. find input sources command-line arguments HTTP requests b. write input generation code completely random inputs unlikely to get very "deep" if method == "GET" ... usually need smart input generator: generate syntactically correct input with random content where freedom is allowed GET /xxx GET /transfer?from=xxx&to=yyy&amount=zzz use test suite to help generate inputs c. execute with random inputs until you get bored maybe some random inputs will trigger an assert that you didn't think an attacker could violate e.g. maybe bank code doesn't check that "from" account has enough balance to cover a transfer; fuzzing may discover relevant arguments, triggering an assert that checks for negative balances or conservation of balances. fuzzers work! they are widely used and have found lots of bugs --they are particularly good at bugs like buffer overflow that may not need a specific input value to trigger them advantages: + better than programmer at testing for unexpected behavior + no need to control entire execution env --can fuzz black-box systems over the network --as long as there's a way to detect errors + may not need source code disadvantages: - uses lots of CPU time - hard to cover everything can miss bugs because didn't happen to try a particular input e.g. if command == "credit" or if inputs must have complex structure e.g. hard to test a compiler with purely random input --symbolic execution: see below 3. Symbolic execution intro more sophisticated testing scheme program "executes" in a different way the symbolic executor builds expressions that relate the program's variables to its original inputs computes inputs that drive program down each side of each "if" NOTE: an assert is an "if" so symbolic executor will try to find inputs that cause each assert to fail What else is an "if"? division: symbolic executor tries to find inputs that make the denominator zero memory reference: symbolic executor tries to find inputs that make the memory reference illegal example: 1. read x, y 2. if x > y: 3. x = y 4. if x < y: 5. x = x + 1 6. assert(x + y == 7) NOTE: line 6 expands into: 6a. if x + y == 7: 6b. error() The idea here is that the programmer knows that x + y should never equal 7, or else something bad happened. Okay, can that line be reached? this is a simple example of code that's hard for humans or fuzzers to test i.e. to decide if attacker could trigger the error requires some careful thought by human requires a lot of luck in a random fuzzer 4. EXE A. Operation x and y hold the example's inputs they hold symbolic values -- not concrete values let's say x=alpha and y=beta EXE remembers which memory locations hold symbolic values and what each location's current symbolic value is EXE remembers "constraints" imposed by executed if statements the "path constraints" (pc) EXE views execution as a tree, which splits at each if EXE executes down one side of each if, then down the other the "if" adds the condition to one execution's pc, and "not" to the other when EXE gets to the error() call, it checks whether the current path constraints can be satisfied EXE uses the STP constraint solver if yes, STP indicates the satisfying values of input variables if attacker inputs those values, program will call error() Let's trace through: one path: 1. pc = { }, x = a, y = b (alpha, beta) 2. pc = { a > b }, x = a, y = b (fork) 3. pc = { a > b }, x = b, y = b 4. same (no fork) 6. cannot satisfy b + b == 7 skips if, continues executing another path: 2. pc = { a <= b }, x = a, y = b 4. pc = { a <= b AND a < b }, x = a, y = b 5. pc = { a <= b AND a < b }, x = a + 1, y = b 6. (a + 1 + b) == 7 for a < b? lots of solutions e.g. a=0 b=6 EXE would report an assert failure w/ inputs e.g. a=0 b=6 symbolic execution is very powerful it looks at the program to figure out bug-provoking inputs! it understands what "x + y == 7" implies about the input consequence: EXE produces an input that triggers a bug. this is useful for convincing skeptical developers that their code can in fact take an adverse path that they were convinced could not be taken. EXE is a C-to-C translator -- it transforms C code, then compiles w/ gcc --------------------------------------------------------------------------- Acknowledgment: The staff of 6.858