HOMEWORK 4 with SOLUTION Fundamental Algorithms, Fall 2002, Professor Yap

Due: Thursday Nov 14, in class

INSTRUCTIONS:

• This homework is related to Professor Siegel’s lecture as well as our lecture on dynamic programming.

• TIP OF THE DAY: It is highly recommended that you rewrite your solution neatly for submission, this time paying attention to missing assumptions, unclear notations, etc. You might be surprised at why you discover.

1. Merging of Two Sorted Lists, (10+10 Points)

Solve parts (a) and (b) of problem 8.6 in page 181 of the CLR textbook.

REMARKS: You may find Stirling’s approximation (p.55) for the factorial function useful. Note that to merge a sorted list of m keys with a sorted list of n items, the standard algorithm takes m+n−1 comparisons. Parts (c) and (d) are not graded, but we ask you to attempt it (we will sketch the solution).

SOLUTION:

(a) Let the input lists be A = (a_{1}, a_{2}, …, a_{n}) and B = (b_{1}, …, b_{n}). There are (2n)! ways to permute all the elements in A and B. Let S be the set of these permutations. E.g., for n = 2, and the two lists are A = (a, a′) and B = (b, b′) then we have S = {aa′bb′, aa′b′b, ab′a′b, b′aa′b, …, b′ba′a}, and |S| = 4! = 24. Now partition S into m = \binom{2n}{n} subsets, S = S_{1} ∪ S_{2} ∪ ⋯ ∪ S_{m}, using the following rule: two permutations are put in the same set if they can be obtained from each other by permuting the a’s among themselves, and permuting the b’s among themselves. For instance, S_{1} can be {aa′bb′, aa′b′b, a′abb′, a′ab′b}. Clearly, |S_{i}| = (n!)^{2} for each i = 1, …, m. Note the following: each set S_{i} has exactly one permutation which is consistent with the original input lists. Hence there are exactly (2n)!/(n!)^{2} = \binom{2n}{n} permutations consistent with the input ordering on A and B.

(b) Let h be the height of the decision tree. By the information theoretic lower bound, h ≥ lg \binom{2n}{n}. Using Stirling’s approximation (equation (3.19) of Text), we have

(\frac{n}{e})^{n} \sqrt{2πn} < n! < 1.09 (\frac{n}{e})^{n} \sqrt{2πn}

for all n ≥ 1, using the fact that e^{1/12} = 1.0869 . . .. Thus

\binom{2n}{n} > \frac{\sqrt{2π2n}(2n/e)^{2n}}{2πn(n/e)^{2n}(1.09)^{2}} > \frac{2^{2n}}{\sqrt{2πn}}

. Taking logs, we get h ≥ 2n−lg(2πn)/2 = 2n−o(n). Thus h is the worst case number of comparisons needed to merge two lists of size n.

2. LCS and Edit Distances (10 Points each part)

Let X = agacgttcgtta and Y = cgactgctgta.

• (i) Compute L(X, Y), which is the length of any longest common subsequence of X, Y. Be sure to show a matrix containing your computation.

• (ii) Using the matrix you made in part (i), draw arrows from each square to indicate where each entry was derived from. Next, use these arrows to come up with a longest common subsequence of X and Y.

• (iii) Compute the edit distance D(X, Y), which was defined in recitation to be the minimum number of Insert/Delete/Replace steps to transform X to Y. Again, please show a matrix with your computation.

• (iv) Using the matrix you made in part (iii), draw arrows on your grid to indicate where each entry was derived from. Next, using these arrows to come up with a sequence of D(X, Y) Insert/Delete/Replace instructions to transform X into Y.

• (v) Let X and Y be strings and A the associated matrix containing the computation record for L(X, Y) (as in part(i)). Describe an algorithm in psuedocode which, given (X, Y, A), computes a longest common subsequence of X and Y.

• (vi) Solve the analogous problem to part (v), but this time for the edit distance problem D(X, Y).

SOLUTION:

(i)

Given that:

L(X, Y) = \{ \begin{matrix}0 & if m = 0 or n = 0 \\ 1 + L(X′, Y′) & if x_{m} = y_{n} \\ max L(X′, Y), L(X, Y′) & if x_{m} ≠ y_{n} \end{matrix} \}

We get the following matrix:

0 0  0  0  0  0  0  0  0  0  0  0  0  0
c 1 ↑ 0 ← 0 0  0  1  1  1  1  1  1  1  1  1
g 2 0  0 ↖ 1 1  1  2  2  2  2  2  2  2  2
a 3 0  1  1 ↖ 2 2  2  2  2  2  2  2  2  3
c 4 0  1  1  2 ↖ 3 3  3  3  3  3  3  3  3
t 5 0  1  1  2 ↑ 3 3  4  4  4  4  4  4  4
g 6 0  1  2  2  3 ↖ 4 4  4  4  5  5  5  5
c 7 0  1  2  2  3 ↑ 4 ← 4 4  5  5  5  5  5
t 8 0  1  2  2  3  4  5 ↖ 5 ← 5 5  6  6  6
g 9 0  1  2  2  3  4  5  5  5 ↖ 6 ← 6 6  6
t 10 0  1  2  2  3  4  5  6  6  6  7 ↖ 7 7
a 11 0  1  2  3  3  4  5  6  6  6  7  7 ↖ 8
0  1  2  3  4  5  6  7  8  9  10 11 12
a  g  a  c  g  t  t  c  g  t  t  a

(ii) Arrows are as shown in previous matrix. Using these arrows, we get an acceptable longest common subsequence of gacgtgta (iii) Given that:

D(X, Y) = \{ \begin{matrix}max |X|, |Y | & if m = 0 or n = 0 \\ D(X′, Y′) & if x_{m} = y_{n} \\ 1 + min D(X′, Y), D(X, Y′), D(X′, Y′) & if x_{m} ≠ y_{n} \end{matrix} \}

We get the following matrix:

0 0 1  2  3  4  5  6  7  8  9  10  11  12
c 1 1 ↖ 1 2  3  3  4  5  6  7  8  9  10  11
g 2 2 2 ↖ ∗1 2  3  3  4  5  6  7  8  9  10
a 3 3 2  2 ↖ ∗1 2  3  4  5  6  7  8  9  9
c 4 4 3  3  2 ↖ ∗1 2  3  4  5  6  7  8  9
t 5 5 4  4  3  ↑ 2  2  2  3  4  5  6  7  8
g 6 6 5  4  4  3 ↖ ∗2 3  3  4  4  5  6  7
c 7 7 6  5  5  4  3 ↖ 3 4  3  4  5  6  7
t 8 8 7  6  6  5  4  3 ↖ ∗3 ← 4 4  4  5  6
g 9 9 8  7  7  6  5  4  4  4 ↖ ∗4 ← 5 5  6
t 10 10 9  8  8  7  6  5  4  5  5  4 ↖ ∗5 6
a 11 11 10  9  8  8  7  6  5  5  6  5  5 ↖ ∗5
0 1  2  3  4  5  6  7  8  9  10  11  12
a  g  a  c  g  t  t  c  g  t  t  a

(iv) Arrows are as shown in previous matrix. (Note: The diagonal moves with asterisks next to them indicate when x_{m} = y_{n}, which is important in generating the instructions.) Using these arrows, we get the following instructions, which if executed from first to last, with transform X into Y:

X.delete(10) X.delete(8) X.replace(6, c) X.insert(t, 4, 5) X.replace(1, c) (Note: I assume that X.insert(t, 4, 5) inserts a t into X between locations 4 and 5.) (v) Here is the psuedocode. Note that the “+” symbol used on strings denotes string concatenation (just like in Java). LCS(X, Y, A) {

i = |X|; j = |Y |; s = “′′;

while ((i > 0)∥(j > 0)) {
if (i == 0) {
j − −;
} else if (j == 0) {
i − −;
} else if (x_{i} == y_{j}) {
s = x_{i} + s;
i − −; j − −;
} else if (max A[i − 1, j], A[i, j − 1] == A[i − 1, j]) {
i − −;
} else {
j − −;
}
}
return s;

} (vi) Here is the psuedocode. Note that the “+” symbol used on strings denotes string concatenation, and we assume integers and characters as implicitly converted to string form when concatenated with other string (just like in Java). And in my psuedocode, we assume s_{k} corresponds to the k_{th} instruction of s, where each instrcution is to be done in order of first to last. ED(X, Y, A) { i =| X |; j =| Y |; k = 1; while ((i > 0) || (j > 0)) {

if (i == 0) { s_{k} = “X.insert(′′+y_{j}+′′, 0, 1)′′; k + +; j − −;

} else if (j == 0) { s_{k} = “X.delete(′′+i+′′)′′; k + +; i − −;

} else if (x_{i} == y_{j}) {

i − −; j − −; } else {

temp = min A[i, j − 1], A[i − 1, j], A[i − 1, j − 1]; if (temp == A[i − 1, j]) {

s_{k} = “X.delete(′′+i+′′)′′;

k + +;
i − −;

} else if (temp == A[i, j − 1]) { s_{k} = “X.insert(′′+y_{j}+′′,′′ +i+′′,′′ +(i + 1)+′′)′′;

k + +;
j − −;

} else { s_{k} = “X.replace(′′+i+′′,′′ +y_{j}+′′)′′;

k + +;
i − −; j − −;

} }

} return s;

}

3. Generalized LCS (15+30+10 Points) Recall the recurrence relation we showed for L(X, Y).

(a) Derive and prove the analogous recurrence for L(X, Y, Z) is the length of any longest common subsequence of three strings X, Y, Z.

(b) Describe how you would organize the computation of L(X, Y, Z) in a systematic way.

(c) Illustrate your solution to (b) by computing L(X, Y, Z) where X = longest, Y = lengthen and Z = elongated.

SOLUTION:

(a)

L(X, Y, Z) = \{ \begin{matrix}0 & if m = 0 or n = 0 or p = 0 \\ 1 + L(X′, Y′, Z′) & if x_{m} = y_{n} = z_{p} \\ max L(X, Y, Z′), L(X′, Y′, Z) & if x_{m} = y_{n} ≠ z_{p} \\ max L(X, Y′, Z), L(X′, Y, Z′) & if x_{m} = z_{p} ≠ y_{n} \\ max L(X′, Y, Z), L(X, Y′, Z′) & if y_{n} = z_{p} ≠ x_{m} \\ max L(X, Y′, Z′), L(X′, Y, Z′), L(X′, Y′, Z) & if x_{m} ≠ y_{n}andy_{n} ≠ z_{p}andx_{m} ≠ z_{p} \end{matrix}

The justification for these formulas are:

If X or Y or Z is the empty string, we know L(X, Y, Z) = 0, hence we have a basis.

When x_{m} = y_{n} = z_{p}, we know for x_{m} that L(X′, Y, Z) ≤ 1+L(X′, Y′, Z′), and similarly for y_{n} and z_{p}. So discarding any of x_{m}, y_{n}, or z_{p} won’t give us a better answer.

When x_{m} = y_{n} ≠ z_{p}, let k = x_{m} = y_{n}, and let l = z_{p}. We note that L(X′, Y, Z′) ≤ 1 + L(X′, Y′, Z′), so there’s no point in keeping x_{m} if we discard y_{n}. (And similarly, there’s no point in keeping y_{n} if we discard x_{m}.) So our only concern is if to keep both x_{m} and y_{n} and discard z_{p}, or if to keep z_{p} and to discard x_{m} and y_{n}. Hence, we get max L(X, Y, Z′), L(X′, Y′, Z) in this case.

The cases for x_{m} = z_{p} ≠ y_{n} and y_{n} = z_{p} ≠ x_{m} are similar to that of x_{m} = y_{n} ≠ z_{p}.

When x_{m} ≠ y_{n} and y_{n} ≠ z_{p} and x_{m} ≠ z_{p}, we’re deciding which of x_{m}, y_{n}, or z_{p} should be kept, hence the max L(X, Y′, Z′), L(X′, Y, Z′), L(X′, Y′, Z) for this case. (Note to reader: If none of x_{m}, y_{n}, or z_{p} gets kept, note that L(X′, Y′, X′) ≤ L(X, Y′, Z′), so we’re fine with formula mentioned above...)

(b)

LCS(X, Y, Z) {

m =| X |; n =| Y |; p =| Z |;
for (j = 0; j <= n ; j + +) {
for (k = 0; k <= p ; k + +) {
A[0, j, k] = 0;
}
}
for (i = 0; i <= m ; i + +) {

for (k = 0; k <= p ; k + +) {

A[i, 0, k] = 0; }

} for (i = 0; i <= m ; i + +) {

for (j = 0; j <= n ; j + +) {

A[i, j, 0] = 0; }

} for (k = 1; k <= p ; k + +) {

for (j = 1; j <= n ; j + +) {

for (i = 1; i <= m ; i + +) {

if ((x_{i} == y_{j})&&(y_{j} == z_{k})) {
A[i, j, k] = 1 + A[i − 1, j − 1, k − 1];
} else if ((x_{i} == y_{j})&&(x_{i}! = z_{k})) {
A[i, j, k] = max A[i, j, k − 1], A[i − 1, j − 1, k];
} else if ((x_{i} == z_{k})&&(x_{i}! = y_{j})) {
A[i, j, k] = max A[i, j − 1, k], A[i − 1, j, k − 1];
} else if ((y_{j} == z_{k})&&(x_{i}! = y_{j})) {
A[i, j, k] = max A[i − 1, j, k], A[i, j − 1, k − 1];
} else {
A[i, j, k] =
max A[i, j − 1, k − 1], A[i − 1, j, k − 1], A[i − 1, j − 1, k];
}
}

} } return A;

} (c) We get the following tables:

z_{0} = \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ e & 2 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ n & 3 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ g & 4 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ t & 5 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ h & 6 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ e & 7 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ n & 8 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{1} = e \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ e & 2 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ n & 3 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ g & 4 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ t & 5 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ h & 6 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ e & 7 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ n & 8 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 1 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{2} = l \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 3 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ g & 4 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ t & 5 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ h & 6 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 7 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 8 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{3} = o \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 3 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ g & 4 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ t & 5 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ h & 6 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 7 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 8 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{4} = n \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 3 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ g & 4 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ t & 5 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ h & 6 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ e & 7 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ n & 8 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{5} = g \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 3 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ g & 4 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ t & 5 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ h & 6 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ e & 7 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ n & 8 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{6} = a \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 3 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ g & 4 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ t & 5 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ h & 6 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ e & 7 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ n & 8 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{7} = t \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ n & 3 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ g & 4 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ t & 5 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ h & 6 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ e & 7 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ n & 8 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{8} = e \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 2 & 2 & 2 \\ n & 3 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ g & 4 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ t & 5 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ h & 6 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ e & 7 & 0 & 1 & 1 & 2 & 3 & 4 & 4 & 4 \\ n & 8 & 0 & 1 & 1 & 2 & 3 & 4 & 4 & 4 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

z_{9} = d \begin{aligned}& 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ l & 1 & 0 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\ e & 2 & 0 & 1 & 1 & 1 & 1 & 2 & 2 & 2 \\ n & 3 & 0 & 1 & 1 & 2 & 2 & 2 & 2 & 2 \\ g & 4 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 3 \\ t & 5 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ h & 6 & 0 & 1 & 1 & 2 & 3 & 3 & 3 & 4 \\ e & 7 & 0 & 1 & 1 & 2 & 3 & 4 & 4 & 4 \\ n & 8 & 0 & 1 & 1 & 2 & 3 & 4 & 4 & 4 \\ & & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ & & & l & o & n & g & e & s & t \end{aligned}

4. Matrix Chain Product (15 Points) Solve the Matrix Chain Product problem for a matrix chain with 8 matrices, with dimension:

(2, 4, 3, 3, 7, 1, 9, 8, 5)

Please organize your work in a half-matrix as illustrated in Figure 15.3 (p.337, CLR). When you are done, please draw the actual parenthesis structure for evaluating this chain. SOLUTION:

12345678 1 2 3 4 5 6 7 8 12345678 1 2 3 4 5 6 7 8^{0} ^{24}^{0} ^{0} ^{0} ^{0} ^{0} ^{0} ^{0}^{36} ^{63} ^{21} ^{63} ^{72} ^{360}^{42} ^{120} ^{30} ^{48} ^{138}^{112}^{84} ^{42} ^{57} ^{117}^{147}^{50} ^{68} ^{138}^{172}^{78} ^{146}^{174}^{126}^{157}^{148}  ^{1} ^{2} ^{3} ^{1} ^{5} ^{5} ^{5}^{2} ^{3} ^{2} ^{5} ^{5} ^{5}^{3} ^{3} ^{5} ^{5} ^{5}^{4} ^{5} ^{5} ^{5}^{5} ^{5} ^{5}^{6} ^{7}^{7}^{0}^{0}^{0}^{0}^{0}^{0}^{0}^{0}

Figure 1: Matrix Chain Product Solution

Note that the matrix entry m_{ij} is computed as

min_{i≤k<j}(m_{ik} + m_{(k+1)j} + r(i) · c(k) · c(j))

where r(i) denotes the number of rows in matrix i and c(j) denotes the number of columns in matrix j. Thus, the actual parenthesis structure is

(M_{1} × ((M_{2}) × (M_{3} × (M_{4} × M_{5})))) × ((M_{6} × M_{7}) × M_{8}).

5. Treaps (10 + 50 Points)

A treap is a binary search tree combined with a heap structure. Read the description of treaps on pages 296-297 in the CLR textbook.

• (i) Construct a Treap out of the following (key, priority) pairs: (12, 2)(15, 6)(5, 1)(7, 3)(2, 4)(17, 5)

• (ii) Do parts (a), (b), (c), and (d) of problem 13-4 on pages 296-298 in the CLR textbook.

Hints:

– For part (b), it might help to look at section 12.4, pages 265 - 268 in the CLR book. Alternatively, you can try the following: Let H(n) denote the height of a binary tree with n nodes. This is a random variable (so you can take expectation, etc). If i is the number of children in the left subtree of the root, then: H(n) = 1 + max H(i), H(n − 1 − i). Hence,

E[H(n)] = 1 + \frac{1}{n} ∑_{i=0}^{n−1} max E[H(i)], E[H(n − 1 − i)].

Then use real induction to prove that for some constants 0 < c_{1} ≤ c_{2} that c_{1}lg(n) ≤ E[H(n)] ≤ c_{2}lg(n).

– For part (c), first give the English description, followed by pseudocode. If necessary, add comments to your pseudocode.

SOLUTION:

(i)

The node (5, 1) should be the root, with (2, 4) and (12, 2) as the left/right children of (5, 1). The nodes (7, 3) and (17, 5) should be the left/right chldren of (12, 2). And (15, 6) should be the left child of (17, 5).

(ii)

(a)

Let’s suppose that k_{1} is the heap key for node x_{1}, and k_{2} is heap key for node x_{2}, and so on... I’ll prove that for any n, a treap of n nodes must be unique. I will do this by Integer Induction.

Basis:

If we just have a treap of one node, that treap is unique.

Induction Hypothsis:

Assume for all k ≤ n − 1 that a treap for k nodes must be unique.

Induction Step:

For the treap with n nodes, consider the node x_{i} with smallest heap key k_{i}. This node x_{i} must always be at the root of the treap (due to minheap properties). We can then use x_{i} to partition the remaining nodes into two sets L, and R, where:

L = {x_{j} : x_{j} < x_{i}}, and R = {x_{j} : x_{j} > x_{i}}

We know by treap property that L consists of all the nodes in the left subtree of x_{i}, and R consists of all the nodes in the right subtree of x_{i}. And, by our induction hypothesis, since | L |≤ n − 1 and | R |≤ n − 1, the left and right subtrees of x_{i} must be unique, and thus the entire treap of n nodes must be unique.

(b)

By varying the heap keys for each node, we are able to obtain binary trees of any design. So, the expected height of a treap is the expected height of a randomly generated binary tree. Let E[H(n)] denote the expected height of a binary tree with n nodes.

First, we’ll show that c_{1} lg n ≤ E[H(n)] for some c_{1}. We note that when we create a complete binary tree (much like that in the answer for homework 3), the height is ⌊lg n⌋, And, for n ≥ 4, we have that:

⌊lg n⌋ ≥ lg n − 1 ≥ \frac{1}{2} lg n

Since in the best case, the height of a binary tree is at least \frac{1}{2}lgn, then we know c_{1} lg n ≤ E[H(n)] where c_{1} ≥ \frac{1}{2}.

Next, we’ll prove that E[H(n)] ≤ c_{2} lg n for some c_{2}.

Using our hint, this means:

E[H(n)] = 1 + \frac{1}{n} ∑_{i=0}^{n−1} max E[H(i)], E[H(n − 1 − i)].

Substituting F (n) = E[H(n)], we have:

F (n) = 1 + \frac{1}{n} ∑_{i=0}^{n−1} max F (i), F (n − 1 − i).

Next, I will use this to show for all n ≥ 1 that F (n) ≤ c_{2} lg n for some constant c_{2}. I’ll prove this by Induction. (I’m assuming F (1) = 0 to make things simpler...)

Basis: F (1) = 0 ≤ c_{2} ∗ lg 1 = 0 Induction Hypothesis: For all k ≤ n − 1, we assume that F (k) ≤ c_{2} ∗ lg k. Induction Step: (Please note that students weren’t responsible for accounting for floors and ceilings in their answers.) Since F (i) is non-decreasing, we have that:

F (n) = 1 + \frac{1}{n} ∑_{i=0}^{n−1} max F (i), F (n − 1 − i)
= 1 + \frac{2}{n} ∑_{i=⌊ \frac{n}{2} ⌋}^{n−1} F (i)
≤ 1 + \frac{2}{n} ∑_{i=⌊ \frac{n}{2} ⌋}^{n−1} c_{2} ∗ lg i
= 1 + \frac{2c_{2}}{n} ∑_{i=⌊ \frac{n}{2} ⌋}^{n−1} lg i

From here, by splitting the summation into two parts, we get:

1 + \frac{2c_{2}}{n} ∑_{i=⌊ \frac{n}{2} ⌋}^{n−1} lg i ≤ 1 + \frac{2c_{2}}{n} [ ∑_{i=⌊ \frac{n}{2} ⌋}^{⌊ \frac{3n}{4} ⌋−1} lg i + ∑_{i=⌊ \frac{3n}{4} ⌋}^{n−1} lg i]
≤ 1 + \frac{2c_{2}}{n} [(⌊\frac{3n}{4}⌋ − 1 − ⌊\frac{n}{2}⌋) lg(\frac{3n}{4}) + (n − 1 − ⌊\frac{3n}{4}⌋) lg n]
≤ 1 + \frac{2c_{2}}{n} [(\frac{n}{4} + 1) lg(\frac{3n}{4}) + (\frac{n}{4} + 1) lg n]
= 1 + \frac{2c_{2}}{n} [(\frac{n}{4} + 1)(lg \frac{3}{4} + lg n) + (\frac{n}{4} + 1) lg n]
= 1 + \frac{2c_{2}}{n} [(\frac{n}{4} + 1)(2 lg n + lg \frac{3}{4})]
= 1 + \frac{2c_{2}}{n} [\frac{n}{2} lg n + 2 lg n + \frac{n}{4} lg \frac{3}{4} + lg \frac{3}{4}]
≤ 1 + \frac{2c_{2}}{n} [\frac{n}{2} lg n + 2 lg n + \frac{n}{4} lg \frac{3}{4}]
= 1 + c_{2} lg n + \frac{4c_{2}}{n} lg n + \frac{c_{2}}{2} lg \frac{3}{4}

And for n ≥ 256, \frac{lg n}{n} ≤ \frac{8}{256} = \frac{1}{32}, therefore:

1 + c_{2} lg n + \frac{4c_{2}}{n} lg n + \frac{c_{2}}{2} lg \frac{3}{4} ≤ 1 + c_{2} lg n + 4c_{2} \frac{1}{32} + \frac{c_{2}}{2} lg \frac{3}{4}
≤ 1 + c_{2} lg n + \frac{1}{8}c_{2} + \frac{c_{2}}{2} lg \frac{3}{4}

And 1 + c_{2} lg n + \frac{1}{8}c_{2} + \frac{c_{2}}{2} lg \frac{3}{4} ≤ c_{2} lg n when:

1 + c_{2} lg n + \frac{1}{8}c_{2} + \frac{c_{2}}{2} lg \frac{3}{4} ≤ c_{2} lg n
1 + \frac{1}{8}c_{2} + \frac{c_{2}}{2} lg \frac{3}{4} ≤ 0
1 + c_{2}(\frac{1}{8} + \frac{1}{2} lg \frac{3}{4}) ≤ 0
1 ≤ −c_{2}(\frac{1}{8} + \frac{1}{2} lg \frac{3}{4})
\frac{1}{−(\frac{1}{8} + \frac{1}{2} lg \frac{3}{4})} ≤ c_{2}

And, \frac{1}{−( \frac{1}{8} + \frac{1}{2} lg \frac{3}{4} )} ≤ 12.2, so we select c_{2} ≥ 12.2 and we’re done!

(Note that −(\frac{1}{8} + \frac{1}{2} lg \frac{3}{4}) > 0) An alternative solution is as follows. If we do Real Induction, which allows us to ignore floors and ceilings, the Basis and Inductive Hypthesis are identical (where δ = 1), and we get as before,

F (n) = 1 + \frac{2c_{2}}{n} ∑_{i= \frac{n}{2}}^{n−1} lg i

And using Integral Approximation from page 1067 of the CLR book,

1 + \frac{2c_{2}}{n} ∑_{i=n/2}^{n−1} lg i ≤ 1 + \frac{2c_{2}}{n} ∫_{n/2}^{n} \frac{lnx}{ln2}dx

= 1 + \frac{2c_{2}}{nln2} ∫_{n/2}^{n} lnxdx
= 1 + \frac{2c_{2}}{nln2}[xlnx − x]_{x=n/2}^{n}
= 1 + \frac{2c_{2}}{n} [x lg x − \frac{x}{ln2}]_{x=n/2}^{n}
= 1 + \frac{2c_{2}}{n} [\frac{n}{2} lg n + \frac{n}{2}(1 − \frac{1}{ln2})]
= 1 + c_{2} lg n + c_{2}(1 − \frac{1}{ln2})

And, 1 + c_{2} lg n + c_{2}(1 − \frac{1}{ln2}) ≤ c_{2} lg n when:

1 + c_{2}(1 − \frac{1}{ln2}) ≤ 0
1 ≤ c_{2}(\frac{1}{ln2} − 1)
\frac{1}{\frac{1}{ln2} − 1} ≤ c_{2}

(Note that \frac{1}{ln2} − 1 > 0. This might not be obvious from what I did...) And \frac{1}{\frac{1}{ln2} −1} ≤ 2.3, so we select c_{2} so that c_{2} ≥ 2.3. (c) To insert a node x, we assign it some random value k which to be its heap key. We then place (x, k) into the treap based on the value of x by regular BST insertion. Then, we rotate (x, k) with its parent until (x, k) is at a position where k is larger than the heap key of the l, the heap key of the parent of (x, k). The rotations assure that (x, k) remains in left/right order by the tree-key with respect to the other nodes. Psuedocode is: TREAP INSERT(x) {

k = random number; BST INSERT(x, k); while (((x, k).parent ≠ NULL) && (k < (x, k).parent heap key)) {

rotate((x, k));

} } (d) Let’s suppose that h is the height of the treap. On average, doing BST INSERT takes time linearly proportional to h. And for randomly chosen k, the number of rotations done on (x, k) afterwards is also linearly proportional to h. Therefore on average, we take time Θ(h). And from part (b), we’ve proven the expected h = lgn. Thus, expected average time is Θ(lgn).