Overview
This course considers the challenge of converting high-level algorithmic ideas into efficient parallel code. We will see how this challenge is greatly simplified by modern programming languages and programming techniques, especially functional programming techniques which make it possible to quickly develop efficient, scalable, and correct implementations. Students will learn how to design, analyze, implement, and evaluate the performance of parallel algorithms across a variety of problem domains (e.g., graph analysis, computational geometry, numerical algorithms, image processing, etc.). In the second half of the course, students will complete a parallel programming project of their own design.
Prerequisites: CSCI-GA.2110 Programming Languages. Familiarity with functional programming and recursive programming techniques is helpful.
Topics include:
- work and span, parallelism
- sequential baselines, work-efficiency, speedup
- parallel functional programming
- parallel algorithm design techniques: divide-and-conquer, contraction, prefix doubling, etc.
- parallel data structures: sequences, sets, tables/dictionaries, graphs, etc.
-
higher-order parallel primitives:
map
,reduce
,filter
,scan
, etc. - determinism and non-determinism
- parallel algorithms from a variety of domains: sorting, searching, order statistics, text/image/audio processing, graph analysis, computational geometry, numerical algorithms, etc.
Schedule
(Note: tentative—subject to change)
Week | Date | Lecture | Notes | Homework | |
---|---|---|---|---|---|
0 | Mon | Jan 20 | no lecture—MLK day (university holiday) | hw1 released | |
1 | Mon | Jan 27 | introduction, parallel hardware, parallelism vs concurrency, problem vs algorithm, sequential baseline, overhead, speedup | ||
2 | Mon | Feb 3 |
nested fork-join parallelism,
par ,
divide-and-conquer,
MPL programming language,
parallel functional programming
|
hw1 due hw2 released |
|
3 | Mon | Feb 10 |
work and span,
language-based cost models,
basic recurrences,
reduce
|
hw2 due hw3 released |
|
4 | Tue | Feb 18 |
(Note: lecture Tue instead of Mon)
contraction, parallel prefix sums, scan ,
filter
|
hw3 due hw4 released |
|
5 | Mon | Feb 24 | parallel sorting and searching, parallel order statistics |
hw4 due hw5 released |
|
6 | Mon | Mar 3 | graphs: undirected and directed, sparse representations, parallel traversals |
hw5 due hw6 released |
|
7 | Mon | Mar 10 | graphs (cont.): parallel traversals, contraction | hw6 due | |
8 | Mon | Mar 17 | the parallel zoo: parallelism in Rust, Java, Go, ISPC, CUDA, Futhark, etc. | project proposals due | |
Fri | Mar 21 | proposal revisions due (if applicable) | |||
9 | Mon | Mar 24 | no lecture—spring break | ||
10 | Mon | Mar 31 | advanced topics: fusion, eliminating intermediate allocation | ||
11 | Mon | Apr 7 | advanced topics: on-the-fly concurrency and non-determinism | ||
12 | Mon | Apr 14 | advanced topics: parallel hashing and hash tables | project checkpoint due | |
13 | Mon | Apr 21 | advanced topics: TBD | ||
14 | Mon | Apr 28 | advanced topics: TBD | ||
15 | Mon | May 5 | project presentations | projects due | |
Thu | May 8 |
(Note: 10:00am–11:50am)
project presentations |
Resources
There is no required textbook. The content of this course is roughly based on the free textbook Algorithms: Parallel and Sequential, by Umut A. Acar and Guy Blelloch.
Policies
Grading: homework assignments (50%), final project (50%)
Deadlines: All deadlines are at 5:00pm (eastern time) on the date listed in the schedule.
Late Submissions: 10% score penalty for each day late. Submissions will be not be accepted if they are submitted more than one week late.
Academic Integrity: Please review the department academic integrity policy. In this course, you are permitted to discuss assignments with other students as long as all discussion adheres to the following "whiteboard policy". Discussion may take place at a whiteboard (or on a scrap of paper, etc.), but no record of the discussion may be kept (all notes must be erased or discarded, no audio or video recording, etc.) and you must allow at least two hours to pass after the discussion before working on the assignment. Being able to recreate any solution from memory is considered proof that you actually understand the solution. If you collaborate with someone in this way on an assignment, you must list their name(s) in your submission. Copying solutions or any other work is a serious offense.