Parallel Computing                       
Prof. Mohamed Zahran   (aka Prof. Z)
mzahran AT cs DOT nyu DOT edu
Tu/Th 9:30-10:45 am  Location: WWH 317
Office Hours (WWH 320): Tue 1-3pm

It is all about parallelism!! 


Welcome students! ... to the Parallel Computing course, edition Spring 2014.  I will keep updating this page regularly. If you have questions related to that course feel free to email me. 

Most of us have learned to program a single microprocessor using a high-level programming language like C/C++, Java, ... This is called sequential programming. We feel very comfortable with this because we think in sequential way and give the machine statements to be executed in sequence. However, this has to change. A single microprocessor (or single core) no longer exists in almost all computers we are using today (including your tablets and smart phones). Most of our devices are now multicore processors. A multicore processor contains several core (called CPUs or cores) on-chip. To make the best use of these multicore chips we need to program them in-parallel. Sequential programming, for all platforms from smartphones to supercomputers, is falling out of fashion and taking back-seat to parallel programming.

How to think in parallel? How to write code in parallel to make the best use of the underlying hardware? How is that new hardware different from the traditional one? What will the future be for the software and hardware? This is the topic of this course.

Here is some basic information about this course:

Midterm exam: March 25th, 2014
Final exam:  May 15th, 2014 8AM-9:50AM (note the time!)

Mailing List

Sign up for the Mailman mailing list for the course. If the system has not already signed you.
You can do so by clicking here. Please follow the mailing list etiquette. 
  • Use the Reply command to contribute to the current thread, but NOT to start another topic.
  • If quoting a previous message,  try to trim off irrelevant parts.
  • Use a descriptive Subject: field when starting a new topic.
  • Do not use one message to ask two unrelated questions.
  • Do NOT make the mistake of sending your completed project assignment to the mailing list!

  • Lecture Notes

    101/28Why Parallel Computing?1.1 - 1.4
    201/30Parallel Hardware: Basics2.1 - 2.2
    302/04Parallel Hardware: Advanced2.3hw1 assigned
    402/06Parallel Hardware: Examples
    502/11Parallel Software: Basics2.4, 2.7hw2 assigned
    602/13Parallel Software: Advanced
    702/18Performance Analysis2.6
    802/20MPI - I3.1
    902/25MPI - II3.3, 3.2, 3.4.1-3.4.5lab1 assigned
    1002/27MPI - III3.4.6-3.5
    1103/04MPI - IV3.7
    1203/06MPI - Last Touch
    1303/11OpenMP -I5.1-5.2
    03/13Revision    Sol
    03/18No class - Spring recess

    03/20No class - Spring recess
    03/25MIDTERM EXAM  Sol
    1403/27OpenMP - II 5.3, 5.4, 5.5
    1504/01OpenMP - III5.6, 5.7
    1604/03OpenMP - IV5.8
    1704/08OpenMP- Last Touch5.9Lab 2 assigned
    1804/10GPUs - Intro
    1904/15CUDA - I
    2004/17CUDA - II
    2104/22CUDA - IIIhw 3 assigned
    2204/24CUDA - IVLab 3 assigned
    2304/29CUDA - V
    2405/01CUDA - VI
    2505/06Final Thoughts
    2605/08Revision (sol)


    Before doing any MPI programming, do the following once you login onto your CIMS account:
    ssh to one of the computational nodes
    cd $HOME
    touch .mpd.conf
    chmod 600 .mpd.conf
    and then use an editor to insert a line like
    (replace xxxxxxx with any secret word of your choice)
    You need to do the above steps only once.
    Then each time you login to your CIMS account type:
    mpd &  
    (Note: you may receive an error message that mpd is already on. In this case you are good to go)

    Lab1 due March 13th
    You will need this skeleton file and this reference program.
    You will also need this reference input ... and this  ... and this!
    Lab2 due April 22nd
    Some test files and their (sub-optimal) solutions: 3cities  (sol), 5cities (sol), 10cities (sol)
    To help you, this program generates random problems with a (sub)optimal solutions. 
    ( usage: ./tsm num    Where num is the number of cities)

    Before doing lab3, please take a look at the following link and follow its steps.
    Lab3 due May 6th

    Homework Assignment 

    hw1  due Feb 11th  sol   comments
    hw2  due Feb 18th  sol   comments
    hw3  due Apr 29th  sol

    Interesting Links (Geeky stuff )

    If you have an interesting link, please email it to the instructor and it will find its way to this page.

    Top500 Supercomputers in the world
    Future Chips (targeting both software and hardware folks interested in parallel programming)

    Designing and Building Parallel Programs

    Multicore info

    The trouble with multicore

    Computer Architecture

    Introduction to parallel computing

    The Landscape of Parallel Computing

    EPCC benchmarks for OpenMP