Introduction to Parallel Programming is an IAP course at MIT.
Parallel computing has been an important research topic in science and technology for decades. Thanks to the fast-developing field of deep learning in recent years, parallel computing is being used for more broad interests. In this class, concepts of parallel computing will be introduced. Attendees will learn not only the basics of high-performance computing (HPC) clusters and GPU accelerators but also programming skills with OpenMP, MPI, CUDA, Pytorch, and Deepspeed. Examples and hands-on exercises will be provided in several programming languages including C, Fortran, and Python. These parallel programming skill sets are useful for researchers to accelerate their computer programs and helpful for students to be prepared for a career in information technology.
Parallel programming for CPU including HPC basics, OpenMP, and MPI.
Parallel programming for GPU including GPU basics, CUDA, and distributed deep leaning (Pytorch, Deepspeed).