Basic Algorithms: Lecture 2

1.1.4 Analyzing Recursive Algorithms

Consider a recursive version of innerProduct. If the arrays are of size 1, the answer is clearly A[0]B[0]. If n>1, we recursively get the inner product of the first n-1 terms and then add in the last term.

Algorithm innerProductRecursive
    Input: Positive integer n and two integer arrays A and B of size n.
    Output: The inner product of the two arrays

if n=1 then
    return A[0]B[0]
return innerProductRecursive(n-1,A,B) + A[n-1]B[n-1]

How many steps does the algorithm require? Let T(n) be the number of steps required.

• If n=1 we do a comparison, two (array) fetches, a product, and a return.
• So T(1)=5.
• If n>1, we do a comparison, a subtraction, a method call, the recursive computation, two array fetches, a product, a sum and a return.
• So T(n) = 1 + 1 + 1 + T(n-1) + 2 + 1 + 1 + 1 = T(n-1)+8.
• This is called a recurrence equation. In general these are quite difficult to solve in closed form, i.e. without T on the right hand side.
• For this simple recurrence, one can see that T(n)=8n-3 is the solution.
• We will learn more about recurrences later.

1.2 Asymptotic Notation

One could easily complain about the specific primitive operations we chose and about the amount we charge for each one. For example, perhaps we should charge one unit for accessing a scalar variable. Perhaps we should charge more for division than for addition. Some computers can multiply two numbers and add it to a third in one operation. What about the cost of loading the program?

Now we are going to be less precise and worry only about approximate answers for large inputs. Thus the rather arbitrary decisions made about how many units to charge for each primitive operation will not matter since our sloppiness (i.e. the approximate forms of our answers) will be valid for any of these (reasonable) choice of primitive operations and the charges assigned to each. Please note that the sloppiness will be very precise.

1.2.1 The Big-Oh Notation

Definition: Let f(n) and g(n) be real-valued functions of a single non-negative integer argument. We write f(n) is O(g(n)) if there is a positive real number c and a positive integer n₀ such that f(n)≤cg(n) for all n≥n₀.

What does this mean?

For large inputs (n≥n₀), f is not much bigger than g (specifically, f(n)≤cg(n)).

Examples to do on the board

1. 3n-6 is O(n). Some less common ways of saying the same thing follow.
2. 3x-6 is O(x).
3. If f(y)=3y-6 and id(y)=y, then f(y) is O(id(y)).
4. 3n-6 is O(2n)
5. 9n⁴+12n²+1234 is O(n⁴).
6. innerProduct is O(n)
7. innerProductBetter is O(n)
8. innerProductFixed is O(n)
9. countPositives is O(n)
10. n+log(n) is O(n).
11. log(n)+5log(log(n)) is O(log(n)).
12. 12345⁵⁴³²¹ is O(1).
13. 3/n is O(1). True but not the best.
14. 3/n is O(1/n). Much better.
15. innerProduct is O(3n⁴+100n+log(n)+34.5). True, but awful.

A few theorems give us rules that make calculating big-Oh easier.

Theorem (arithmetic): Let d(n), e(n), f(n), and g(n) be nonnegative real-valued functions of a nonnegative integer argument and assume d(n) is O(f(n)) and e(n) is O(g(n)). Then

1. ad(n) is O(f(n)) for any nonnegative a
2. d(n)+e(n) is O(f(n)+g(n))
3. d(n)e(n) is O(f(n)g(n))

Theorem (transitivity): Let d(n), f(n), and g(n) be nonnegative real-valued functions of a nonnegative integer argument and assume d(n) is O(f(n)) and f(n) is O(g(n)). Then d(n) is O(g(n)).

Theorem (special functions): (Only n varies)

1. If f(n) is a polynomial of degree d, then f(n) is O(n^d).
2. n^k is O(aⁿ) for any k>0 and a>1.
3. log(n^k) is O(log(n)) for any k>0
4. (log(n))^k is O(n^j) for any k>0 and j>0.

Example: (log n)¹⁰⁰⁰ is O(n^0.001). This says raising log n to the 1000 is not (significantly) bigger than the thousandth root of n. Indeed raising log to the 1000 is actually significantly smaller than taking the thousandth root since n^0.001) is not O((log n)¹⁰⁰⁰).

So log is a VERY small (i.e., slow growing) function.

Homework: R-1.19 R-1.20