G63.2011.002/G22.2945.001, Fall 2010:

High Performance Scientific Computing

Marsha Berger
Andreas Klöckner
Tues, 5:10-7:00 pm. 101 Warren Weaver Hall.

Course Description

This class will introduce the student to the fundamentals of parallel scientific computing. We will first establish an understanding of different types of machines from the point of view of large-scale floating-point-heavy workloads. This will include a study of CPU and GPU architectures, interconnects, and forms of parallel memory. The practicalities of programming these machines (MPI, OpenMP, OpenCL/CUDA) will be introduced, accompanied by homework assignments for each of these three major approaches. Issues such as load balancing, communication, and synchronization overhead will be addressed throughout, and established practice in the field in the form of parallel numerical algorithms will be studied. Since a prerequisite for good parallel performance is good serial performance, this aspect of high-performance computing will also be addressed . Along the way important tools for scientific computing will be emphasized, including for example debuggers, Makefiles, version control systems.

This will be a hands-on class, with several parallel (and serial) computing assignments. There will be a final project at the end. Students who have code they want to parallelize are encouraged to attend, and use that for their final project.

Prerequisites for the course are (serial) programming experience with C/C++ or FORTRAN and some familiarity with numerical methods. You will learn Unix for this course, if you don't know if already.