Example: T(N) = T(N/4) + 2N
Example: T(N) = 9T(N/3) + N2.5
Example: T(N) = 9T(N/3) + N2
Example: T(N) = 2T(N/2) + rN (our original problem)
We want to multiply big integers.
When we multiply without machine help, we use as knowledge the times and addition tables for all 1 digit numbers. In some sense our ``internal computer'' has as primitive operations the multiplication and addition of two 1-digit number. To multiply larger numbers, we apply the 5th-grade algorithm that requires Θ(N2) steps to multiply two N-digit numbers.
Computers typically have a single instruction to multiply two 32-bit numbers (or 64-bit on some machines). One way to enable a computer to multiply two 32N-bit numbers would be to implement the 5-th grade algorithm in software (using 32-bit numbers as ``digits'') and again compute the result in Θ(N2) time.
To make the wording easier let's consider the ``human'' case where the primitive operations deal with 1-bit numbers and we want to multiply two N-bit numbers X and Y. We also assume that N is a power of 2. As we said above, using the 5th grade algorithm we can perform the multiplication in time Θ(N2). We want to go faster.
If we simply divide the bits in half (the high order N/2, and the low order N/2), we see that to multiply X and Y we need to compute 4 sub-products Xhi*Yhi, Xhi*Ylo, Xlo*Yhi, Xlo*Ylo. The 5th grade algorithm specifies that we must put them in the correct "column" and add.
Xhi Xlo x Yhi Ylo ----------------------- Xhi*Ylo Xlo*Ylo Xhi*Yhi Xlo*Yhi
Since addition of K-bit numbers is Θ(K) using the 3rd grade algorithm, and our multiplications are of N/2-bit values we get
T(N) = 4T(N/2)+cnWe now excitedly apply the master theorem (case 1). Unfortunately the result is T(N)=Θ(N2), which is no improvement. We try again and with cleverness are able to get by with three instead of four multiplications.
We compute Xhi*Yhi and Xlo*Ylo as before, but then compute the miraculous (Xhi-Xlo)*(Ylo-Yhi). The miracle multiplication produces the two remaining terms we need, but has added to this two other terms. However, however those terms are just the (negative of) the two terms we did compute so we can add them back and they cancel.
Do this on the board.
The summary is that T(N) = 3T(N/2)+cN. (This c is larger than the one we had before.) Now the master theorem (case 1) gives
T(N) = Θ(Nlog23N)
Since log23<1.585, we get the surprising
Theorem: We can multiple two N-bit numbers in o(N1.585) time.