We have seen that the fastest comparison-based sorting algorithms run in time Θ(Nlog(N)), where N is the number of items to sort. In this section we are going to develop faster algorithms. Hence they must not be comparison-based algorithms.

We make a key assumption, we are sorting items whose keys are integers in a bounded range [0, R-1].

**Question**: Let's start with a special case R=N so
we are sorting N items with integer keys from 0 to N-1. As usual
we assume there are no duplicate keys. Also as usual we remark that
it is easy to lift this restriction. How should we do this sort?

**Answer**: That was a trick question. You don't even
have to look at the input. If you tell me to sort 10 integers in the
range 0...9 and there are no duplicates, I know the answer is
{0,1,2,3,4,5,6,7,8,9}. Why? Because, if there are no duplicates, the
input must consist of one copy of each integer from 0 to 10.

OK, let's drop the assumption that R=N. So we have N items (k,e), with each k an integer, no duplicate ks, and 0≤k<R. The trick is that we can use k to decide where to (temporarily) store e.

Algorithm preBucketSort(S) input: A sequence S of N items with integer keys in range [0,N) output: Sequence S sorted in increasing order of the keys. let B be a vector of R elements, each initially a special marker indicating empty while (not S.isEmpty()) (k,e) ← S.remove(S.first()) B[k] ← e <==== the key idea (not a comparison) for i ← 0 to R-1 do if (B[i] not special marker) then S.insertLast((i,B[i])

To convert this algorithm into bucket sort we drop the artificial assumption that there are not duplicates. Now instead of a vector of items we need a vector of buckets, where a bucket is a sequence of items.

Algorithm BucketSort(S) input: A sequence S of N items with integer keys in range [0,N) output: Sequence S sorted in increasing order of the keys. let B be a vector of R sequences of items each initially empty while (not S.isEmpty()) (k,e) ← S.remove(S.first()) B[k].insertLast(e) <==== the key idea for i ← 0 to R-1 do while (not B[i].isEmpty()) S.insertLast((i,B[i].remove(B[i].first())))

The first loop has N iterations each of which run in time
Θ(1), so the loop requires time Θ(N).
The for loop has R iterations so the while statement is
executed R times. Each while statement requires time Θ(1)
(excluding the body of the while) so all of them require time Θ(N)
The **total** number of
iterations of all the inner while loops is again N and each again
requires time Θ(1), so the time for all
inner iterations is Θ(N).

The previous paragraph shows that the complexity is Θ(N)+Θ(R) = Θ(N+R).

So bucket-sort is a winner if R is not too big. For example if R=O(N), then bucket-sort requires time only Θ(N). Indeed if R=o(Nlog(N)), bucket-sort is (asymptotically) faster than any comparison based sorting algorithm (using worst case analysis).

**Definition**: We call a sort **stable**
if equal elements remain in the same relative position. Stated more
formally: for any two items (k_{i},e_{i}) and
(k_{j},e_{j}) such that item
(k_{i},e_{i}) precedes item
(k_{j},e_{j}) in S (i.e., i<j), then item
(k_{i},e_{i}) precedes item
(k_{j},e_{j}) after sorting as well.

Stability is often convenient and will prove so in the next section on radix-sort. We note that bucket-sort is stable since we treated each bucket in a fifo manner inserting at the rear and removing from the front.

Let's extend our sorting study from integer keys to keys that are
themselves pairs of integers. The first question to ask is, given two
keys (k,m) and (k',m'), which is larger? Note that (k,m) is just the
**key**; an item would be written ((k,m),e).

**Definition**:
The **lexicographical** (dictionary) ordering on pairs of
integers is defined by declaring (k,m) < (k',m') if
either

- k < k' or
- k = k' and m < m'

Note that this really is dictionary order:

canary < eagle < egret < heron

Algorithm radix-sort-on-pairs input: A sequence S of N items with keys pairs of integers in the range [0,N) Write elements of S as ((k,m),e) output: Sequence S lexicographically sorted on the keys bucket-sort(S) using m as the key bucket-sort(S) using k as the key

Do an example of radix sorting on pairs and then do it incorrectly, by reversing the order of the bucket sorts.

What if the keys are triples or in general d-tuples?

The answer is ...

**Homework:** R-4.15

**Theorem**: Let S be a sequence of N items each of
which has a key (k_{1},k_{2},...k_{d}), where
k_{i} is in integer in the range [0,R). We can sort S
lexicographically in time O(n(N+R)) using radix-sort.

Insertion sort or bubble sort are not suitable for general sorting
of large problems because their running time is quadratic in N, the
number of items. For small problems, when time is not an issue, these
are attractive because they are so simple.
Also if the input is almost sorted, insertion sort is fast since it
can be implemented in a way that is O(N+A), where A is the number of
**inversions**, (i.e., the number of pairs out of order).

Heap-sort is a fine general-purpose sort with complexity Θ(Nlog(N)), which is optimal for comparison-based sorting. Also heap-sort can be executed in place (i.e., without much extra memory beyond the data to be sorted). (The coverage of in-place sorting was ``unofficial'' in this course.) If the in-place version of heap-sort fits in memory (i.e., if the data is less than the size of memory), heap-sort is very good.

Merge-sort is another optimal Θ(Nlog(N)) sort. It is not easy to do in place so is inferior for problems that can fit in memory. However, it is quite good when the problem must be do ``out-of-core'' since it doesn't fit. We didn't discuss this issue, but the merges can be done with two input and one output file (this is not trivial to do well, you want to utilize the available memory in the most efficient manner).

Quick-sort is hard to evaluate. The version with the fast median algorithm is fine theoretically (worst case again Θ(Nlog(N)) but not used because of large constant factors in the fast median. Randomized quick-sort has a low expected time but a poor worst-case time. It can be done in place and is quite fast in that case, often the fastest. But the quadratic worst case is a fear (and a non-starter for many real-time applications).

Bucket and radix sort are wonderful when they apply, i.e., when the
keys are integers in a modest range (R a small multiple of N). For
radix sort with d-tuples the complexity is Θ(d(N+R)) so if
d(N+R) is o(Nlog(N)), radix sort is asymptotically faster than
**any** comparison based sort (e.g., heap-, insertion-,
merge-, or quick-sort).

Selection means the ability to find the kth smallest element. Sorting will do it, but there are faster (comparison-based) methods. One example problem is finding the median (N/2 th smallest).

It is not too hard (but not easy) to implement selection with linear expected time. The surprising and difficult result is that there is a version with linear worst-case time.