================ Start Lecture #1 ================

Basic Algorithms

2002-03 Fall
Monday and Wednesday 11-12:15
Ciww 102

Chapter 0: Administrivia

I start at 0 so that when we get to chapter 1, the numbering will agree with the text.

0.1: Contact Information

0.2: Course Web Page

There is a web site for the course. You can find it from my home page, which is http://allan.ultra.nyu.edu/~gottlieb

0.3: Textbook

The course text is Goodrich and Tamassia: ``Algorithm Design: Foundations, Analysis, and Internet Examples.

0.4: Computer Accounts and Mailman Mailing List

0.5: Grades

The major components of the grade will be the midterm, the final, and homeworks. I will post (soon) the weights for each.

0.6: Midterm

We will have a midterm. As the time approaches we will vote in class for the exact date. Please do not schedule any trips during days when the class meets until the midterm date is scheduled.

0.7: Homeworks and Labs

If you had me for 202, you know that in systems courses I also assign labs. Basic algorithms is not a systems course; there are no labs. The homeworks, which are problem sets, are required, will be graded, and will constitute a portion of your grade. Very few if any homeworks will require the computer

0.8: Recitation

There is a recitation session on tuesdays from 9:30 to 10:45 in room 109. The recitation leader is Sean McLaughlin <seanmcl@cs.nyu.edu>.

0.9: Obtaining Help

Good methods for obtaining help include

  1. Asking me during office hours (see web page for my hours).
  2. Asking the mailing list.
  3. Asking another student, but ...
    Your homeworks must be your own.

0.10: The Upper Left Board

I use the upper left board for homework assignments and announcements. I should never erase that board. Viewed as a file it is group readable (the group is those in the room), appendable by just me, and (re-)writable by no one. If you see me start to erase an announcement, let me know.

0.11: A Grade of ``Incomplete''

It is university policy that a student's request for an incomplete be granted only in exceptional circumstances and only if applied for in advance. Naturally, the application must be before the final exam.

Part I: Fundamental Tools

Chapter 1: Algorithm Analysis

We are interested in designing good algorithms (a step-by-step procedure for performing some task in a finite amount of time) and good data structures (a systematic way of organizing and accessing data).

Unlike v22.102, however, we wish to determine rigorously just how good our algorithms and data structures really are and whether significantly better algorithms are possible.

1.1: Methodologies for Analyzing Algorithms

We will be primarily concerned with the speed (time complexity) of algorithms.

We will emphasize instead and analytic framework that is independent of input and hardware, and does not require an implementation. The disadvantage is that we can only estimate the time required.

Homework: Unless otherwise stated homework problems are from the last section in the current book chapter. R-1.1 and R-1.2.

1.1.1: Pseudo-Code

Designed for human understanding. Suppress unimportant details and describe some parts in natural language (English in this course).

1.1.2: The Random Access Machine (RAM) Model

The key difference from reality is the assumption of a very simple memory model: Accessing any memory element takes a constant amount of time. This ignores caching and paging for example. It also ignores the word-size of a computer (any size number can be stored in one word and accessed in one operation time).

The time required is simply a count of the primitive operations executed. Primitive operations include

  1. Assign a value to a variable (independent of the size of the value; but the variable must be a scalar).
  2. Method invocation, i.e., calling a function or subroutine.
  3. Performing a (simple) arithmetic operation (divide is OK, logarithm is not).
  4. Indexing into an array (for now just one dimensional; scalar access is free).
  5. Following an object reference.
  6. Returning from a method.

1.1.3: Counting Primitive Operations

Let's start with a simple algorithm (the book does a different simple algorithm, maximum).

Algorithm innerProduct
    Input: Non-negative integer n and two integer arrays A and B of size n.
    Output: The inner product of the two arrays

prod ← 0
for i ← 0 to n-1 do
    prod ← prod + A[i]*B[i]
return prod

The total is thus 1+1+5n+(n+1)+1 = 6n+4.

Let's improve it (a very little bit)

Algorithm innerProductBetter
    Input: Non-negative integer n and two integer arrays A and B of size n.
    Output: The inner product of the two arrays

prod ← A[0]*B[0]
for i ← 1 to n-1 do
    prod ← prod + A[i]*B[i]
return prod

The cost is 4+1+5(n-1)+n+1 = 6n+1


If n=0, we access A[0] and B[0], which do not exist. The original version returns zero as the inner product of empty arrays, which is arguably correct. The best fix is perhaps to change Non-negative to Positive. Let's call this algorithm innerProductBetterFixed.

What about if statements?

Algorithm countPositives
    Input: Non-negative integer n and an integer array A of size n.
    Output: The number of positive elements in A

pos ← 0
for i ← 0 to n-1 do
    if A[i] > 0 then
        pos ← pos + 1
return pos

Let U be the number of updates done.

1.1.4: Analyzing Recursive Algorithms

Consider a recursive version of innerProduct. If the arrays are of size 1, the answer is clearly A[0]B[0]. If n>1, we recursively get the inner product of the first n-1 terms and then add in the last term.

Algorithm innerProductRecursive
    Input: Positive integer n and two integer arrays A and B of size n.
    Output: The inner product of the two arrays

if n=1 then
    return A[0]B[0]
return innerProductRecursive(n-1,A,B) + A[n-1]B[n-1]

How many steps does the algorithm require? Let T(n) be the number of steps required.