lecture4

DATA STRUCTURES AND ALGORITHMS CS 3139
Lecture: March 25th, 1999

ANNOUNCEMENTS

None for today

REVIEW

Sorting Algorithms and Issues

Insertion sort
Bubble sort
Selection sort
Lower bound for simple sorting
Shell sort

We had a lively, circus-like demonstration of the various insertion algorithms. Thanks to all brave and eager "volunteers" who participated.

TODAY

More Shell sort

Picking increments

Heapsort
Mergesort
Quicksort!

SHELL SORT

Choosing increments

1st proposed by Shell: h_t= floor(N/2), h_k= floor(h_k+1/2)

We provided the code that implements Shellsort. Please see figure 7.4 in the book for the code.

Analysis of Shellsort: The first sorting algorithm to perforn < O(N^2)

However, it is hard to prove the < O(N^2) for general conditions
Worst case Shell sequence can result in O(N^2)
There exists inputs that take Omega(N^2) to run
Show the upper bound
The following sequence is a worst case scenario: 5 1 6 2 7 3 8 4

Perform pass with increment h_k
₌h_k insertions sorts of N/h_k elements
    ==> Pass cost O(h_k(N/h_k)^2) = O(N^2/h_k)
Sum over all passes
    O(Sum from i = 1 of N^2/h_i)
    = O(N^2(Sum from i = 1 of 1/h_i)
    = O(N^2(1+1/2+1/4+...1/(N/2))
    = O(N^2-2) = O(N^2)

An alternative way to select increments

Hibbard Sequence: 1, 3, 7, ... 2^k-1 - O(N^(3/2))
Sedgewick Sequence: O(N^(4/3))

Shell sort is very interesting from a theoritician's point of view. In practice, other, more-efficient, algorithms are more common.

HEAPSORT

Build heap O(N): insert all elements in random order, percolate down on each non-leaf (parent X < X)
N deletemin ==> store in 2nd array - O(NlogN)
Copy elements back - O(N)
Overall time complexity - O(NlogN)

Problems

2x storage needs
O(N) extra storage needed
Sorted, but backwards! Thus, use DeleteMax instead - new ordering property: parent X > X
O(NlogN) worst & average case performance

We went through a Heapsort example in class.

MERGESORT

O(NlogN) worst case performance
O(N) extra storage

Example: Merge 2 sorted lists A and B to get C
    A:    1    13    24    26
    B:    2     5     27    38
    C:    1    2    5    13    24    26    27    38             O(N-1) comparisons

Overall Algorithm

If N==1, done
else, mergesort first half, mergesort second half, merge sorted halves

We went through a mergesort example.

Mergesort is a classic example of a Divide and Conquer algorithm.
Mergesort requires copying ==> O(N) extra storage

Mergesort Analysis

Assume N is a power of 2, for simplicity of spliting
N == 1, mergesort runs in constant time O(1)
T(N): mergesort(N/2)*2 + merge = 2T(N/2) + N
T(N)/N = T(N/2)/(N/2) + 1
T(N/2)/(N/2) = T(N/4)/(N/4) + 1
T(N/4)/(N/4) = T(N/8)/(N/8) + 1
...
T(2)/2 = T(1)/T(1) + 1
T(N)/N = T(1)/1 + logN
T(N) = O(NlogN)

QUICKSORT

Fastest known algorithm in practice
Average running time is still O(NlogN), but the constant factor is small
Worst case O(N^2), but is very unlikely
Works best for large values of N
For smaller N's, Quicksort isn't as good,
If N is samll (<20), do something simpler, e.g. use insertion sort

General idea behind Quicksort

If N==0 or 1, return
Pick pivot V cleverly, where V is an element of the set to be sorted
We generate 2 sets L1 and L2: L1 = All elements smaller than V, L2 = All elements larger than V
return Quicksort(S1), V, Quicksort(S2)

d Nhat Minh Dau, nmd13@columbia.edu