Homework 2 Fundamental Algorithms, Fall 2002, Professor Yap

Due: Thu Oct 3, in class

INSTRUCTIONS:

• TIP OF THE DAY: Your solution should be both LOGICAL (of course) and GRAMMATICAL (what?). Some students think that mathematical writing can ignore grammar, especially if you use some symbols. This is not true. Grammatical means it is constructed like a normal English writing: your solution is a sequence of complete ENGLISH sentences (even if your sentence involves mathematical symbols equations)! That means you begin each sentence with a capital letter, and end each sentence with a full stop. You may think the last point is trivial, until you try to grade a solution where you do not know where one sentence stops and the next begins!

1. (10 Points) The Fibonacci recurrence is

F (n) = F (n − 1) + F (n − 2).

(i) Suppose ϕ = \frac{1+\sqrt{5}}{2} = 1.61803... (this is also called the golden ratio) and ϕ̂ = 1 − ϕ = −0.61803.... Show that ϕ and ϕ̂ are solutions of the equation x^{2} = x + 1 SOLUTION:

The roots of the equation ax^{2} + bx + c = 0 can be obtained using the

formula x = ^{\frac{−b±\sqrt{(b^{2}−4ac)}}{2a}}. Thus, the roots of the equation x^{2} − x − 1 = 0 are given by \frac{1±\sqrt{1+4}}{2} i.e. ϕ and ϕ̂.

(ii) Suppose the initial conditions are F (0) = a and F (1) = b. Find α, β such that

F (n) = αϕ^{n} + βϕ̂^{n}

is a solution.

SOLUTION:

We note that

• F (0) = α + β = a.

• F (1) = αϕ + βϕ̂ = b.

We now have a pair of linear equations in the two variables α and β and they can be easily calculated to be α = \frac{b−a_{ϕ}}{ϕ−_{ϕ}} and β = \frac{aϕ−b}{ϕ−_{ϕ}}.

2. (20 Points) Consider the sum T (n) = ∑_{i=1}^{n} f(i) in the following cases: (i) f(i) = i^{3} (ii) f(i) = 1/i (iii) f(i) = i/ log i (iv) f(i) = i!

(v) f(i) = \frac{2^{i}}{i^{2}} (vi) f(i) = (log i)^{log i} In each case, say if T (n) is a polynomial-type sum, an exponential-type sum or neither. You must prove your claim. When it is polynomial-type or exponential-type sum, state the Θ-order of T (n).

SOLUTION:

For every function we need to first check if it’s decreasing or increasing. In case it’s decreasing, we can eliminate the test for polynomial-type sums. Then, we can apply the tests required for exponential-type sums.

The tests are given here for reference

• Polynomial type f(n) is increasing and f(n) = O(f(\frac{n}{2})). In this case, T (n) = Θ(n.f(n)).

• Exponential type (A) There exists a constant C > 1 such that for all i, f(i) ≥ C · f(i − 1). In this case, T (n) = Θ(f(n)), or (B) There exists a constant c < 1 such that for all i, f(i) ≤ c·f(i−1). In this case, T (n) = Θ(f(1)).

(i) f(i) = i^{3}

SOLUTION:

We note that f(i) is increasing with i and so we test for polynomial type.

We note that i^{3} ≤ C · (\frac{i}{2})^{3} for all C ≥ 8. So, f(i) = O(f(\frac{i}{2})). Hence, T (n) = Θ(n · n^{3}) = Θ(n^{4}).

(ii) f(i) = \frac{1}{i}.

SOLUTION:

We note that f(i) = \frac{1}{i} is decreasing and hence is not a polynomial type sum. So, we test for the second case in exponential type sums i.e.,

\begin{aligned}& \frac{1}{i} & ≤ & c_{0} · \frac{1}{i−1} & for some c_{0} < 1 \\ ⇒ & \frac{i−1}{i} & ≤ & c_{0} \end{aligned}

But this condition doesn’t hold for i > i_{0} where i_{0} = \frac{1}{1−c_{0}} . Thus, we conclude that f(n) is neither polynomial-type nor exponential-type.

(iii) f(i) = i/ log i.

SOLUTION:

We note that f(i) is increasing and so test for polynomial-type.

We have to show that \frac{i}{log i} ≤ C · \frac{\frac{i}{2}}{log \frac{i}{2}} for some C > 0. We now have

\frac{\frac{i}{2}}{log \frac{i}{2}} = \frac{1}{2} · \frac{i}{log i−1}
> \frac{1}{2} · \frac{i}{log i}

This implies that \frac{i}{log i} ≤ 2 · \frac{\frac{i}{2}}{log \frac{i}{2}} . Thus, T (n) = Θ(n · \frac{n}{log n}) = Θ(\frac{n^{2}}{log n}). (iv) f(i) = i!. SOLUTION: We note that f(i) is increasing and guess that it’s exponential-type. We proceed to make test (A) for exponential-type sums. We need to show that i! ≥ C · (i − 1)! for some C > 1. But this holds for all i > 2 and hence f(n) grows exponentially. Hence T (n) = Θ(n!).

(v) f(i) = \frac{2^{i}}{i^{2}} . SOLUTION: We note that f(n) is increasing as \frac{\frac{2^{i}}{i^{2}}}{\frac{2^{i−1}}{(i−1)^{2}}} = 2 · (\frac{i−1}{i})^{2} > 1. We guess f(i) grows exponentially large and make test (A). We need to show that \frac{2^{i}}{i^{2}} ≥ C · \frac{2^{i−1}}{(i−1)^{2}} for some C > 1 i.e., 2 · (\frac{i−1}{i})^{2} > C. lim_{i→∞} \frac{i−1}{i} = 1 and hence we have to choose 1 < C < 2. Thus, f(i) grows exponentially large and hence T (n) = O(\frac{2^{n}}{n^{2}} ). (vi) f(i) = (log i)^{log i}. SOLUTION: We note that f(i) is increasing and show that it’s neither polynomial-type nor exponential-type. Assume that f(i) is polynomial-type, i.e., there exists a c > 0 such that (log i)^{log i} ≤ c · (log \frac{i}{2})^{log ^{\frac{i}{2}}} for i large enough. We will derive a contradiction. Taking logarithm of both sides, we get

log c + (log(i/2) log log(i/2) > log i · log log i
⇒  log c + (log(i/2) log log i > log i · log log i
⇒  log c > log log i · (log i − log(i/2))
⇒  log c > log log i · log 2

This is a contradiction since the R.H.S → ∞ as i → ∞. This proves that f(i) is not polynomial-type. Before we prove that (log i)^{log i} is not exponential, we note a very useful fact, namely, log i−log(i−1) = Θ(\frac{1}{i}). To see this, note that log i−log(i− 1) = Θ(H(i) − H(i − 1)) = Θ(\frac{1}{i}). For exponential type, we need to show that there exists a C > 1 such that (log i)^{log i} ≥ C · (log (i − 1))^{log (i−1)}. Taking logarithms of both sides, we get

log i · log log i  ≥ log C + log (i − 1) · log log (i − 1)
log i · log log i − log (i − 1) · log log (i − 1)  ≥ log C
log (i − 1)(log log i − log log(i − 1)) + Θ(\frac{log log i}{i}) ≥ log C > 0

The term Θ(\frac{log log i}{i}) arises from (log i−log(i−1)) log log i. This term goes to 0 as i → ∞. Next consider the term log (i − 1)(log log i−log log (i − 1)). We have log log i − log log (i − 1) = Θ(∫_{log (i−1)}^{log i} \frac{dx}{x}) = Θ(\frac{log i−log (i−1)}{log (i−1)}) obtained by replacing \frac{1}{x} with the largest value \frac{1}{log (i−1)} in the integral. So the

term log (i − 1)(log log i − log log (i − 1)) = log (i − 1) · Θ(\frac{log i−log (i−1)}{log (i−1)}) = Θ(\frac{1}{i}) → 0 as i → ∞. Thus, we have shown that the LHS goes to 0 but the RHS is positive (since C > 1). This is a contradiction.

In summary, we have shown that f(i) is neither polynomial-type nor exponential-type.

3. Exercise 1.1 in the handout on Master Theorem. NOTE: no proofs are necessary for this question.

State the solution, up to Θ− order of the following recurrences:

(i) T (n) = 10T (\frac{n}{10}) + log^{10} n. (ii) T (n) = 100T (\frac{n}{10}) + n^{10}. (iii) T (n) = 10T (\frac{n}{100}) + (log n)^{log log n}. (iv) T (n) = 16T (\frac{n}{4}) + 4^{lg n}.

SOLUTION:

We now state the Master theorem and then read off the Θ− order of the recurrences by stating the applicable case:

Theorem 1 Let a ≥ 1 and b > 1 be constants, let f(n) be a function and let T (n) be defined on the nonnegative integers by the recurrence T (n) = aT (\frac{n}{b}) + f(n).

where we interpret \frac{n}{b} to mean either ⌊\frac{n}{b}⌋ or ⌈\frac{n}{b}⌉. Then T (n) can be bounded asymptotically as follows:

(a) If f(n) = O(n^{log_{b} a−ϵ}) for some constant ϵ > 0, then T (n) = Θ(n^{log_{b} a}).

(b) If f(n) = Θ(n^{log_{b} a}), then T (n) = Θ(n^{log_{b} a} lg n).

(c) If f(n) = Ω(n^{log_{b} a+ϵ}) for some constant ϵ > 0, and if af(\frac{n}{b}) ≤ cf(n) for some constant c < 1 and all sufficiently large n, then T (n) = Θ(f(n)).

(i) T (n) = 10T (\frac{n}{10}) + log^{1} 0n.

SOLUTION:

We note that f(n) = log^{10} n = O(n^{log_{10} 100−ϵ}) and hence case (1) applies and so T (n) = Θ(n). (ii) T (n) = 100T (\frac{n}{10}) + n^{10}. SOLUTION: We note that f(n) = n^{10} and n^{log_{b} a} = n^{2}. We see that n^{1}0 = Ω(n^{2+ϵ}) and also, 100f(\frac{n}{10}) = \frac{n^{10}}{10^{8}} ≤ \frac{1}{10^{8}} · n^{10} = c · f(n). Thus taking c = \frac{1}{10^{8}} < 1 works. Hence, we note that case 2 applies and T (n) = Θ(n^{10}). (iii) T (n) = 10T (\frac{n}{100}) + (log n)^{log log n}. SOLUTION: Here, we need to compare (log n)^{log log n} and n^{\frac{1}{2}} . Thus, we have

(log n)^{log log n} ≤ n^{\frac{1}{2} −ϵ}
⟺ (log log n)^{2}  ≤ (\frac{1}{2} − ϵ) · log n

which is true as log^{k} n = O(n^{c}) for all k, c > 0. Thus, case 1 applies and T (n) = Θ(n^{\frac{1}{2}} ). (iv) T (n) = 16T (\frac{n}{4}) + 4^{lg n}. SOLUTION: We have f(n) = 4^{lg n} = n^{2} = n^{log^{4} 16}. So, case 2 appleis and we get T (n) = Θ(n^{2} log n).

4. Exercise 1.3 in the handout on Master Theorem. Solve the master recurrence when f(n) = w(n) log^{k} n, for any k. SOLUTION: We have T (n) = aT (\frac{n}{b}) + f(n). We make a domain transformation by substituting n with the form b^{m} and t(m) = T (b^{m}). Then, we get t(m) = at(m−1)+f(b^{m}). We now make a range transformation by dividing with a^{m} and setting s(m) = \frac{t(m)}{a^{m}}. We then obtain s(m) = s(m − 1) + \frac{f(b^{m})}{a^{m}}. We set s(0) = 0 and obtain s(m) = ∑_{i=1}^{m} \frac{w(b^{i}) log^{k} b^{i}}{a^{i}} where n = b^{m}. Now,

∑_{i=1}^{m} \frac{w(b^{i}) log^{k} b^{i}}{a^{i}} = ∑_{i=1}^{m} \frac{a^{i} log^{k} b^{i}}{a^{i}}
= ∑_{i=1}^{m} log^{k} b^{i}

First, we assume that k ≥ 0 . Then, we observe that this is a polynomialtype sum and hence ∑_{i=1}^{m} log^{k} b^{i} = Θ(l · log^{k} b^{m}) = Θ(log b^{m} · log^{k} b^{m}) = Θ(log^{k+1} n). If k = −1, we have ∑_{i=1}^{m} log^{−1} (b^{i}) = log^{−1} (b) · ∑_{i=1}^{m} \frac{1}{i} = log^{−1} (b) · Θ(log m) = Θ(log log n) Finally, if k < −1, we have ∑_{i=1}^{m} log^{k} b^{i} = Θ ∑_{i=1}^{m} \frac{1}{i^{k}} = Θ(1). Using the fact that we note that a^{m} = w(n), we have

T (n) = t(b^{m}) = s(m).(a^{m}) = \{ \begin{matrix}Θ(w(n) · log^{k+1} n) & if & k ≥ 0 \\ Θ(w(n) · log log n) & if & k = −1 \\ Θ(w(n)) & if & k < −1 \end{matrix} .

You should get most of the credit if you solved the case for k ≥ 0 correctly. 6