CSCI-GA.3033-121 (Special Topic)
Programming Parallel Algorithms
Fall 2026

Overview

This course considers the challenge of converting high-level algorithmic ideas into efficient parallel code. We will see how this challenge is greatly simplified by modern programming languages and programming techniques, especially functional programming techniques which make it possible to quickly develop efficient, scalable, and correct implementations. Students will learn how to design, analyze, implement, and evaluate the performance of parallel algorithms across a variety of problem domains (e.g., graph analysis, computational geometry, numerical algorithms, image processing, etc.). In the final weeks of the course, students will complete a parallel programming project of their own design.

Prerequisites: CSCI-GA.2110 Programming Languages. Familiarity with functional programming and recursive programming techniques is helpful.

Topics include:

• work and span, parallelism
• sequential baselines, work-efficiency, speedup
• parallel functional programming
• parallel algorithm design techniques: divide-and-conquer, contraction, prefix doubling, etc.
• parallel data structures: sequences, sets, tables/dictionaries, graphs, etc.
• higher-order parallel primitives: map, reduce, filter, scan, etc.
• determinism and non-determinism
• parallel algorithms from a variety of domains: sorting, searching, order statistics, text/image/audio processing, graph analysis, computational geometry, numerical algorithms, etc.

Schedule

(Note: tentative—subject to change)

Week	Date		Lecture	Notes	Homework
0	Mon	Sep 7	no lecture—Labor day (university holiday)
1	Mon	Sep 14	introduction, parallel hardware, parallelism vs concurrency, parallel functional programming, MaPLe, par, scheduling, (self-)speedup	Recommended Reading: APS Ch 2 Sec 1,2 Ch 7 Ch 8	hw1 released
2	Mon	Sep 21	work and span, language-based cost model, recurrences, divide-and-conquer, reduce	Recommended Reading: APS Ch 2 Sec 3 Ch 26 Sec 1 Ch 28 Sec 4	hw1 due hw2 released
3	Mon	Sep 28	work efficiency, recurrences (cont.), contraction, parallel prefix sums, scan, sequences	Recommended Reading: Blelloch93 lib docs: Seq	hw2 due hw3 released
4	Mon	Oct 5	filter, flatten, tree flattening, sorting, parallel divide-and-conquer merging	Recommended Reading: APS Ch 36	hw3 due hw4 released
5	Mon	Oct 12	no lecture—fall break (university holiday)
	Wed	Oct 14	(Note: lecture Wed instead of Mon—legislative Monday) 2D computational geometry, parallel convex hull (quickhull), nearest neighbors	code (nn)	hw4 due hw5 released
6	Mon	Oct 19	graphs: undirected and directed, sparse representations, parallel traversals	Recommended Reading: APS Ch 52 Ch 54 Ligra
7	Mon	Oct 26	versioned sequences, inject, graphs (cont.), edge contraction, star contraction, counting connected components	Recommended Reading: APS Ch 22 Ch 60-63	hw5 due hw6 released
8	Mon	Nov 2	the parallel zoo: parallelism in Go, Java, C++, Rust, etc.	Links: - ParlayLib - Parallel ML Bench - Rayon (Rust)	hw6 due hw7 released
9	Mon	Nov 9	on-the-fly concurrency, compare-and-swap, hashing, hash tables, lock freedom	CollectSort.sml CollectHash.sml HashTable.sml	hw7 due
10	Mon	Nov 16	scheduling by work-stealing	AroraBlumofePlaxton98 AcarBlellochBlumofe00 ChaseLev05 ParlayLib scheduler.h	project proposals due
	Fri	Nov 20			proposal revisions due (if applicable)
11	Mon	Nov 23	pseudo-randomness, splittable pseudorandom number generators, pedigrees and DPRNG, random shuffling	ParlayLib random.h DotMix SplitMix
12	Mon	Nov 30	trees, treaps, joining, interval trees, parallel augmented maps	IntervalTree.sml Just Join PAM	project checkpoint
13	Mon	Dec 7	dynamic programming, bottom-up scheduling, subset sum, seam carving	Recommended Reading: APS Ch 48-49 MaPLe subset sum MaPLe seam carving blog post
14	Mon	Dec 14	fusion, avoiding unnecessary allocations and intermediate data structures		project reports due
15	TBD		(Note: finals week—date and time TBD) in-person project presentations

Policies

Grading: homework assignments (70%), final project (30%)

Deadlines: All deadlines are at 5:00pm (eastern time) on the date listed in the schedule.

Late Submissions: 10% score penalty for each day late. Submissions will be not be accepted if they are submitted more than one week late.

Academic Integrity: Please review the department academic integrity policy. In this course, you are permitted to discuss assignments with other students as long as all discussion adheres to the following "whiteboard policy". Discussion may take place at a whiteboard (or on a scrap of paper, etc.), but no record of the discussion may be kept (all notes must be erased or discarded, no audio or video recording, etc.) and you must allow at least two hours to pass after the discussion before working on the assignment. Being able to recreate any solution from memory is considered proof that you actually understand the solution. If you collaborate with someone in this way on an assignment, you must list their name(s) in your submission. Copying solutions or any other work is a serious offense.

Accommodations: If you are in need of accommodations due to a disability or otherwise, please contact the instructor: s (dogoodt) we!stricluckk (a!t) nyrobotsu (do!t) ed!u

Project

In the final weeks of the course, students will complete a self-directed programming project, responsible for 30% of their overall grade. The project can be completed individually or in groups of two. The goal of the project is to develop a parallel application which achieves real parallel speedups.

Project proposals will be due later in the semester (please see the schedule). Each project must be organized around an explicitly identified problem-based algorithmic challenge: a well-defined computational problem which will be the focus of the project. At a minimum, every project must include the following four components. All parts are mandatory.

1. A clear input-output specification of the problem, including a description of how the inputs and outputs are represented/encoded and how to check whether or not the output is correct.
2. A checker, which you will use to verify the correctness of the implementations you compare.
3. A suitable sequential baseline, i.e., a sequential implementation that solves the problem correctly and efficiently. This will be used to evaluate speedup.
4. A parallel implementation, which aims to achieve real parallel speedups over the sequential baseline.

The checker and/or the sequential baseline can be found online / taken off-the-shelf if you would prefer. The parallel implementation must be your own work.

For the parallel programming component of the project, you must use a dynamically scheduled parallel programming language or library, preferably based on structured fork-join parallelism. All of the following are acceptable. If you wish to use any other language or library, you must first discuss with the instructor and get approval. (Note that Python will not be approved.)

• MaPLe
• ParlayLib (C++)
• Rayon (Rust)
• Go
• Fork/Join Java

We encourage thinking about incorporating your parallel component into a larger application. Note however that the project will be primarily evaluated on the parallel programming component alone.

Project Proposal Guidelines

The proposal should be exactly 1 page, and should clearly state the following information:

• Who is working on the project. Groups of two are encouraged, or you may complete the project individually.
• A brief introduction and motivation for the project (1-2 paragraphs). If your project is inspired by existing research or projects available online, please cite them and describe how your project is related.
• A clear description of the problem-based algorithmic challenge. Explicitly identify the computational problem which will be the focus of the project. Describe the problem carefully. What is the input? What is the output? What data structures will be involved?
• A plan for each of the four mandatory components: the input-output specification, the checker, the sequential baseline, and the parallel implementation. For the checker and the sequential baseline, state whether you plan to implement these yourself or use an off-the-shelf implementation (either is fine). Remember that the parallel implementation must be your own work. How do you plan to evaluate the performance of your solution?
• Project timeline. Please clearly state two sets of goals:
  1. 2-week “checkpoint goal”: everything you hope to complete in the first two weeks of the project.
  2. The final goals of the project: everything you plan to complete before the end of the semester.
• Tools, languages, libraries, resources, etc. Please briefly describe what programming languages/libraries/tools you plan to use and what computing resources you will use to complete the project. (Please see https://cims.nyu.edu/dynamic/systems/resources/computeservers/ for a list of computing resources provided by Courant; we recommend using these for your projects.)

Note that, while working on the project, you might realize that your original stated goals are not attainable. This is okay, and is a normal aspect of research. In this case, you can reach out to the instructor to plan an alternative goal during the course of the project.

MaPLe (MPL)

Homework assignments will use the MaPLe programming language, a high-level parallel programming language which offers a number of features making it simpler and safer to write efficient parallel code. The MaPLe language is based on Standard ML. Students do not need to already be familiar with MaPLe or Standard ML; we will introduce these as part of the course.

Resources

Brightspace. We'll use Brightspace for questions and discussions outside of lecture, and for course announcements, etc.

Gradescope. Homeworks will be submitted and graded on Gradescope.

Textbook. There is no required textbook. The content of this course is roughly based on the free textbook Algorithms: Parallel and Sequential, by Umut A. Acar and Guy Blelloch.

Learning MaPLe. We will introduce this language as part of the course. If you would like to get a head start, we recommend taking a look at mpl-tutorial, especially the first few sections (Hello World, Parallelism and Granularity Control, and Trees). A number of programming examples are available here and here. MaPLe is based on Standard ML. To familiarize yourself with the syntax we recommend this guide.

Compute Servers. Courant has a number of compute servers available for students to use for assignments, research, etc. For this course we recommend students use the following machines, each of which has 32 cores (64 threads) and 256GB of memory.

• crunchy1.cims.nyu.edu
• crunchy2.cims.nyu.edu
• crunchy5.cims.nyu.edu
• crunchy6.cims.nyu.edu

Information about accessing these servers is available here. You will need a CIMS account. If you do not already have a CIMS account, please follow the instructions here. We recommend adding the following to your local SSH configuration, replacing YOUR_CIMS_USERNAME with your CIMS account name. This is usually the same as your NYU NetID.

Host cims-access
  HostName access.cims.nyu.edu
  User YOUR_CIMS_USERNAME
Host cims-crunchy1
  ProxyCommand ssh cims-access nc crunchy1.cims.nyu.edu 22
  User YOUR_CIMS_USERNAME

You can similarly add configurations for the other machines, crunchy2.cims.nyu.edu, etc. You should then be able to log into a crunchy server like so:

$ ssh cims-crunchy1