# Basic Algorithms: Lecture 22

================ Start Lecture #22 ================

We again consider an easy variant of a well known, but difficult, optimization problem.

• We have a set T of N tasks, each with a start time si and a finishing time fi (si<fi).
• Each task must start at time si and will finish at time fi.
• Each task is executed on a machine Mj.
• A machine can execute only one task at a time, but can start a task at the same time as the current task ends (red lines on the figure to the right).
• Tasks are non-conflicting if they do not conflict, i.e., if fi≤sj or fj≤si. For example, think of tasks as college classes.
• The problem is to schedule all the tasks in T using the minimal number of machines.

In the figure there are 6 tasks, with start times and finishing times (1,3), (2,5), (2,6), (4,5), (5,8), (5,7). They are scheduled on three machines M1, M2, M3. Clearly 3 machines are needed as can be seen by looking at time 4.

Note that a good solution to this problem has three objectives that must be met.

1. We must generate a feasible solution. That is, all the tasks assigned to a single machine are non-conflicting. A schedule that does not satisfy this condition is invalid and an algorithm that produces invalid solutions is wrong.

2. We want to generate a feasible solution using the minimal number of machines. A schedule that uses more than the minimal number of machines might be called non-optimal or inefficient and an algorithm that produces such schedules might be called poor.

3. We want to generate optimal solutions in the (asymptotically) smallest amount of time possible. An algorithm that generates optimal solutions using more time might be called slow.

Let's illustrate the three objectives with following example consisting of four tasks having starting and stopping times (1,3), (6,8), (2,5), (4,7). It is easy to construct a wrong algorithm, for example

```Algorithm wrongTaskSchedule(T)
Input:  A set T of tasks, each with start time si and
finishing time fi (si≤fi).
Output: A schedule of the tasks.

while T is not empty do
remove from T the first task and call it i.
schedule i on M1
```

When applied to our 4-task example, the result is all four tasks assigned to machine 1. This is clearly infeasible since the last two tasks conflict.

It is also not hard to produce a poor algorithm, one that generates feasible, but non-optimal solutions.

```Algorithm poorTaskSchedule(T):
Input:  A set T of tasks, each with start time si and
finishing time fi (si≤fi).
Output: A feasible schedule of the tasks.

m ← 0                         { current number of machines }
while T is not empty do
remove from T a task i
m ← m+1
schedule i on Mm
```

On the 4-task example, poorTaskSchedule puts each task on a different machine. That is certainly feasible, but is not optimal since the first and second task can go on one machine.

Hence it looks as though we should not put a task on a new machine if it can fit on an existing machine. That is certainly a greedy thing to do. Remember we are minimizing so being greedy really means being stingy. We minimize the number of machines at each step hoping that will give an overall minimum. Unfortunately, while better, this idea does not give optimal schedules. Let's call it mediocre.

```Algorithm mediocreTaskSchedule(T):
Input:  A set T of tasks, each with start time si and
finishing time fi (si≤fi).
Output: A feasible schedule of the tasks of T

m ← 0                         { current number of machines }
while T is not empty do
remove from T a task i
if there is an Mj having all tasks non-conflicting with i then
schedule i on Mj
else
m ← m+1
schedule i on Mm
```

When applied to our 4-task example, we get the first two tasks on one machine and the last two tasks on separate machines, for a total of 3 machines. However, a 2-machine schedule is possible, as we will shall soon see.

The needed new idea is to processes the tasks in order. Several orders would work, we shall order them by start time. If two tasks have the same start time, it doesn't matter which one is put ahead. For example, we could view the start and finish times as pairs and use lexicographical ordering. Alternatively, we could just sort on the first component.

#### finalAlgorithm

```Algorithm taskSchedule(T):
Input:  A set T of tasks, each with start time si and
finishing time fi (si≤fi).
Output: An optimal schedule of the tasks of T

m ← 0                         { current number of machines }
while T is not empty do
remove from T a task i with smallest start time
if there is an Mj having all tasks non-conflicting with i then
schedule i on Mj
else
m ← m+1
schedule i on Mm
```

When applied to our 4-task example, we do get a 2-machine solution: the middle two tasks on one machine and the others on a second. But is this the minimum? Certainly for this example it is the minimum; we already saw that a 1-machine solution is not feasible. But what about other examples?

#### Correctness (i.e. Minimality of m)

Assume the algorithm runs and declares m to be the minimum number of machines needed. We must show that m are really needed.

• Consider the step when the algorithm increases m to its final value and assume the task under consideration is i.
• At this point the current task conflicts with one (or more) task(s) in each of the m-1 machines currently used.
• But all these tasks have start time no later than si since the tasks were processed in order of their start time.
• Since they conflict with i, they each have finishing time after si.
• Hence they all conflict with each other as well (consider time si).
• Hence we really do need m machines.

OK taskSchedule is feasible and optimal. But is it slow? That actually depends on some details of the implementation and it does require a bit of cleverness to get a fast solution.

#### Complexity

Let N be the number of tasks in T. The book asserts that it is easy to see that the algorithm runs in time O(NlogN), but I don't think this is so easy. It is easy to see O(N2).

• The while loop has N iterations.
• If we initially sort the tasks based on their start time (a one time cost of Θ(NlogN)), the removal is constant time per iteration or Θ(N) time in total.
• To figure out the condition part of the if is not trivial.
• A simple method would be to compare the current task with all previously scheduled tasks, which shows that the iteration is O(N) and the algorithm is O(N2).
• To get O(log(N)) for each iteration, keep the machines in a heap using as key the latest finishing time assigned to that machine. This tells you when that machine will be free (remember that all tasks assigned so far start no later than si, the current job's start time).
• Check the min element of the heap. If it is free at si, then it is free forever starting at si. We now
1. Remove the machine from the heap (removeMin), Θ(logN).
2. Assign the current job to the removed machine, Θ(1).
3. Now this machine is free at fi and we re-insert it into the heap, Θ(logN).
• If it is not free at si, then no machine is free at si so
1. Increase m generating a new machine, Θ(1).
2. Assign i to the new machine m, Θ(1).
3. Insert machine m (which has key fi) into the heap, Θ(logN).

Homework: R-5.3

Problem Set 4, Problem 3.
Part A. C-5.3 (Do not argue why your algorithm is correct).
Part B. C-5.4.

## 5.2 Divide-and-Conquer

The idea of divide and conquer is that we solve a large problem by solving a number of smaller problems and then we apply this idea recursively.

• Merge Sort and Quick Sort are divide and conquer algorithms.

• A common example would be that to solve a single problem of size N we need to solve two problems each of size approximately N/2.

• When we apply the recursion again, we wind up with four problems each of size approximately N/4.

• In addition to the time required to solve the subproblems, we need to include the time to split the original problem into pieces and the time required to combine the solutions to the subproblems into a solution for the original problem.

### 5.2.1 Divide-and-Conquer Recurrence Equations

From the description above we see that the complexity of a divide and conquer solution has three parts.

1. The time required to split the problem into subproblems.
2. The time required to solve all the subproblems.
3. The time required to combine the subproblem solutions

Let T(N) be the (worst case) time required to solve an instance of the problem having time N. The time required to split the problem and combine the subproblems is also typically a function of N, say f(N).

More interesting is the time required to solve the subproblems. If the problem has been split in half then the time required for each subproblem is T(N/2).

Since the total time required includes splitting, solving both subproblems, and combining we get.

```    T(N) = 2T(N/2)+f(N)
```

Very often the splitting and combining are fast, specifically linear in N. Then we get

```    T(N) = 2T(N/2)+rN
```
for some constant r. (We probably should say ≤ rather than =, but we will soon be using big-Oh and friends so we can afford to be a little sloppy.)

What if N is not divisible by 2? We should be using floor or ceiling or something, but we won't. Instead we will be assuming for recurrences like this one that N is a power of two so that we can keep dividing by 2 and get an integer. (The general case is not more difficult; but is more tedious)

But that is crazy! There is no integer that can be divided by 2 forever and still give an integer! At some point we will get to 1, the so called base case. But when N=1, the problem is almost always trivial and has a O(1) solution. So we write either

```           r            if N = 1
T(N) =
2T(N/2)+rN   if N > 1
```
or
```    T(1) = r
T(N) = 2T(N/2)+rN   if N > 1
```

No we will now see three techniques that, when cleverly applied, can solve a number of problems. We will also see a theorem that, when its conditions are met, gives the solution without our being clever.

#### The Iterative Substitution Method

Also called ``plug and chug'' since we plug the equation into itself and chug along.

```    T(N) = 2T(N/2)+rN
= 2[    ]+rN
= 2[2T((N/2)/2)+r(N/2)]+rN
= 4T(N/4)+2rN    now do it again
= 8T(N/8)+3rN
```

A flash of inspiration is now needed. When the smoke clears we get

```    T(N) = 2iT(N/2i)+irN
```

When i=log(N), N/2i=1 and we have the base case. This gives the final result

```    T(N) = 2log(N)T(N/2log(N))+irN
= N     T(1)       +log(N)rN
= N     r          +log(N)rN
= rN+rNlog(N)
```
Hence T(N) is O(Nlog(N))

#### The Recursion Tree

The idea is similar but we use a visual approach.

We already studied the recursion tree when we analyzed merge-sort. Let's look at it again.

The diagram shows the various subproblems that are executed, with their sizes. (It is not always true that we can divide the problem so evenly as shown.) Then we show that the splitting and combining that occur at each node (plus calling the recursive routine) only take time linear in the number of elements. For each level of the tree the number of elements is N so the time for all the nodes on that level is Θ(N) and we just need to find the height of the tree. When the tree is split evenly as illustrated the sizes of all the nodes on each level go down by a factor of two so we reach a node with size 1 in logN levels (assuming N is a power of 2). Thus T(N) is Θ(Nlog(N)).

#### The Guess and Test Method

This method is really only useful after you have practice in recurrences. The idea is that, when confronted with a new problem, you recognize that it is similar to a problem you have seen the solution to previously and you guess that the solution to the new problem is similar to the old.

You then plug your guess into the recurrence and test that it works. For example if we guessed that the solution of

```    T(1) = r
T(N) = 2T(N/2)+rN   if N > 1
```
was
```    T(N) = rN+rNlog(N)
```
we would plug it in and check that
```    T(1) = r
T(N) = 2T(N/2)+rN   if N > 1
```

But we don't have enough experience for this to be very useful.

Allan Gottlieb