CSCI-GA.3033-121 (Special Topic)
Programming Parallel Algorithms
Spring 2025


60 Fifth Ave, Room C15
Mondays 10:15am–12:15pm
Instructor: Sam Westrick
Office Hours: TBD

Overview

This course considers the challenge of converting high-level algorithmic ideas into efficient parallel code. We will see how this challenge is greatly simplified by modern programming languages and programming techniques, especially functional programming techniques which make it possible to quickly develop efficient, scalable, and correct implementations. Students will learn how to design, analyze, implement, and evaluate the performance of parallel algorithms across a variety of problem domains (e.g., graph analysis, computational geometry, numerical algorithms, image processing, etc.). In the second half of the course, students will complete a parallel programming project of their own design.

Prerequisites: CSCI-GA.2110 Programming Languages. Familiarity with functional programming and recursive programming techniques is helpful.

Topics include:

Schedule

(Note: tentative—subject to change)

Week Date Lecture Notes Homework
0 Mon Jan 20 no lecture—MLK day (university holiday) hw1 released
1 Mon Jan 27 introduction, parallel hardware, parallelism vs concurrency, problem vs algorithm, sequential baseline, overhead, speedup
2 Mon Feb 3 nested fork-join parallelism, par, divide-and-conquer, MPL programming language, parallel functional programming hw1 due
hw2 released
3 Mon Feb 10 work and span, language-based cost models, basic recurrences, reduce hw2 due
hw3 released
4 Tue Feb 18 (Note: lecture Tue instead of Mon)
contraction, parallel prefix sums, scan, filter
hw3 due
hw4 released
5 Mon Feb 24 parallel sorting and searching, parallel order statistics hw4 due
hw5 released
6 Mon Mar 3 graphs: undirected and directed, sparse representations, parallel traversals hw5 due
hw6 released
7 Mon Mar 10 graphs (cont.): parallel traversals, contraction hw6 due
8 Mon Mar 17 the parallel zoo: parallelism in Rust, Java, Go, ISPC, CUDA, Futhark, etc. project proposals due
Fri Mar 21 proposal revisions due (if applicable)
9 Mon Mar 24 no lecture—spring break
10 Mon Mar 31 advanced topics: fusion, eliminating intermediate allocation
11 Mon Apr 7 advanced topics: on-the-fly concurrency and non-determinism
12 Mon Apr 14 advanced topics: parallel hashing and hash tables project checkpoint due
13 Mon Apr 21 advanced topics: TBD
14 Mon Apr 28 advanced topics: TBD
15 Mon May 5 project presentations projects due
Thu May 8 (Note: 10:00am–11:50am)
project presentations

Resources

There is no required textbook. The content of this course is roughly based on the free textbook Algorithms: Parallel and Sequential, by Umut A. Acar and Guy Blelloch.

Policies

Grading: homework assignments (50%), final project (50%)

Deadlines: All deadlines are at 5:00pm (eastern time) on the date listed in the schedule.

Late Submissions: 10% score penalty for each day late. Submissions will be not be accepted if they are submitted more than one week late.

Academic Integrity: Please review the department academic integrity policy. In this course, you are permitted to discuss assignments with other students as long as all discussion adheres to the following "whiteboard policy". Discussion may take place at a whiteboard (or on a scrap of paper, etc.), but no record of the discussion may be kept (all notes must be erased or discarded, no audio or video recording, etc.) and you must allow at least two hours to pass after the discussion before working on the assignment. Being able to recreate any solution from memory is considered proof that you actually understand the solution. If you collaborate with someone in this way on an assignment, you must list their name(s) in your submission. Copying solutions or any other work is a serious offense.