Basic Algorithms

================ Start Lecture #24 ================

Notes

We did all the examples in recitation so will go quickly.
The final will be in room 303 EDUC. Where is EDUC?

Example: T(N) = T(N/4) + 2N

a=1, b=4, and f(N)=2N
log_ba = log₄1 = 0
N^log_b = N⁰ = 1
We have case 3 since f(N) is Ω(N^0+ε) for &epsilon=1 and af(N/b) = (1)2(N/4) = N/2 = (1/4)f(N)
Hence T(N) = Θ(N)

Example: T(N) = 9T(N/3) + N^2.5

a=9, b=3, f(N)=N^2.5
log_ba = 2, so N^log_ba = N²
We have case 3 since f(N) is Ω(N^0+ε) for &epsilon=.5 and af(N/b)=9f(N/3)= 9(N/3)^2.5=(1/3)^0.5f(n).
Hence T(n) is O(N^2.5)

Example: T(N) = 9T(N/3) + N²

a=9, b=3, f(N)=N²
log_ba = 2, so N^log_ba = N²
We have case 2 with k=0 since f(N) is Θ(N^log_ba).
Hence T(n) is Θ(N²log(N))

Example: T(N) = 2T(N/2) + rN (our original problem)

a=2, b=2, f(N)=rN
log_ba = 1, so N^log_ba = N¹ = N.
We have case 2 with k=0 since f(N) is Θ(N^log_ba).
Hence T(n) is Θ(Nlog(N))

Homework: R-5.4

5.2.2 Integer Multiplication

We want to multiply big integers.

When we multiply without machine help, we use as knowledge the times and addition tables for all 1 digit numbers. In some sense our ``internal computer'' has as primitive operations the multiplication and addition of two 1-digit number. To multiply larger numbers, we apply the 5th-grade algorithm that requires Θ(N²) steps to multiply two N-digit numbers.

Computers typically have a single instruction to multiply two 32-bit numbers (or 64-bit on some machines). One way to enable a computer to multiply two 32N-bit numbers would be to implement the 5-th grade algorithm in software (using 32-bit numbers as ``digits'') and again compute the result in Θ(N²) time.

To make the wording easier let's consider the ``human'' case where the primitive operations deal with 1-bit numbers and we want to multiply two N-bit numbers X and Y. We also assume that N is a power of 2. As we said above, using the 5th grade algorithm we can perform the multiplication in time Θ(N²). We want to go faster.

If we simply divide the bits in half (the high order N/2, and the low order N/2), we see that to multiply X and Y we need to compute 4 sub-products Xhi*Yhi, Xhi*Ylo, Xlo*Yhi, Xlo*Ylo. The 5th grade algorithm specifies that we must put them in the correct "column" and add.

                Xhi Xlo
             x  Yhi Ylo
-----------------------
        Xhi*Ylo Xlo*Ylo
Xhi*Yhi Xlo*Yhi

Since addition of K-bit numbers is Θ(K) using the 3rd grade algorithm, and our multiplications are of N/2-bit values we get

   T(N) = 4T(N/2)+cn

We now excitedly apply the master theorem (case 1). Unfortunately the result is T(N)=Θ(N²), which is no improvement. We try again and with cleverness are able to get by with three instead of four multiplications.

We compute Xhi*Yhi and Xlo*Ylo as before, but then compute the miraculous (Xhi-Xlo)*(Ylo-Yhi). The miracle multiplication produces the two remaining terms we need, but has added to this two other terms. However, however those terms are just the (negative of) the two terms we did compute so we can add them back and they cancel.

Do this on the board.

The summary is that T(N) = 3T(N/2)+cN. (This c is larger than the one we had before.) Now the master theorem (case 1) gives

T(N) = Θ(N^log₂3N)

Since log₂3<1.585, we get the surprising

Theorem: We can multiple two N-bit numbers in o(N^1.585) time.

Homework: R-5.5