Awesome Parallel Computing Resources

A curated list of awesome parallel computing resources. Please feel free to update this page through submitting pull requests or emailing me.

Table of Contents

• Software
• Presentations
• Learning Resources
• Datasets
• Links

All the lists in this page are either in alphabetical order or chronological order.

Software

Programming Library

• CAF: An Open Source Implementation of the Actor Model in C++
• CGraph: A cross-platform DAG framework based on C++17
• Chapel: A Programming Language for Productive Parallel Computing on Large-scale Systems
• Cilk Plus: C/C++ Extension for Data and Task Parallelism
• Taskflow: A Modern C++ Parallel Task Programming Library
• FastFlow: High-performance Parallel Patterns in C++
• Galois: A C++ Library to Ease Parallel Programming with Irregular Parallelism
• Heteroflow: Concurrent CPU-GPU Task Programming using Modern C++
• HPX: A C++ Standard Library for Concurrency and Parallelism
• Intel TBB: Threading Building Blocks
• Kokkos: A C++ Programming Model for Writing Performance Portable Applications on HPC platforms
• OpenMP: Multi-platform Shared-memory Parallel Programming in C/C++ and Fortran
• RaftLib: A C++ Library for Enabling Stream and Dataflow Parallel Computation
• STAPL: Standard Template Adaptive Parallel Programming Library in C++
• STLab: High-level Constructs for Implementing Multicore Algorithms with Minimized Contention
• Transwarp: A Header-only C++ Library for Task Concurrency
• Workflow: C++ Parallel Computing and Asynchronous Networking Engine

Concurrent Data Structures

• Boost.Lockfree: Thread-safe and Lock-free Containers
• ConcurrentQueue: A Fast Multi-producer Multi-consumer Lock-free Concurrent Queue in C++
• libcds: A C++ library of Concurrent Data Structures
• Parallel-Hashmap: A Header-only Very Fast and Memory-friendly Hash Map
• ReaderWriterQueue: A Fast Single-producer Single-consumer Lock-free Queue in C++
• xenium: A C++ library Providing Various Concurrent Data Structures and Reclamation Schemes

Coroutine and Fiber

• Boost.Fiber: A Framework for Userland-threads Programming and Scheduling
• cppcoro: A Library for C++ Coroutines Abstractions for the Coroutines TS
• Fiber Tasking Lib: A Library for Enabling Task-based Multi-threading using Fibers
• coro-async: A lite C++ coroutine TS based networking library

Presentations

Generic Parallel Computing Topics

• Concurrency in C++20 and Beyond - A. Williams
• Is Parallel Programming still Hard? - P. McKenney, M. Michael, and M. Wong at CppCon 2017
• The Speed of Concurrency: Is Lock-free Faster? - Fedor G Pikus in CppCon 2016

Scheduling in Parallel Processing

• Expressing Parallelism in C++ with Threading Building Blocks - Mike Voss at Intel Webinar 2018
• A Work-stealing Runtime for Rust - Aaron Todd in Air Mozilla 2017

Memory Model

• C++11/14/17 atomics and memory model: Before the story consumes you - Michael Wong in CppCon 2015
• The C++ Memory Model - Valentin Ziegler at C++ Meeting 2014

Learning Resources

Books

• C++ Concurrency in Action: Practical Multithreading - Anthony Williams 2012
• The Art of Multiprocessor Programming - Maurice Herlihy 2012
• Parallel Computing: Theory and Practice - Umut A. Acar 2016

Tutorials

• Parallel Computing Training Tutorials - Lawrence Livermore National Laboratory

Position Papers

• The Landscape of Parallel Computing Research: A View from Berkeley
• Extreme Heterogeneity 2018: Productive Computational Science in the Era of Extreme Heterogeneity

Experimental Papers

• Evaluation and Analysis of Dynamic Loop Scheduling in OpenMP

Courses

• CS6290 High-performance Computer Architecture - Milos Prvulovic and Catherine Gamboa at George Tech

Datasets

• HPEC Graph Challenge
• PARRSEC Benchmark Suite for the Analysis of Multithreaded Programs

Links

Blogs

• 1024 Cores - Dmitry Vyukov
• Michael Wong's Standard on Parallelism and Programming Languages - Michael Wong
• Preshing on Programming - Jeff Preshing
• Sutter's Mill - Herb Sutter
• The Black Art of Concurrency - Internal Pointers

Journals

• IEEE Transactions on Parallel and Distributed Systems (TPDS)
• Journal of Parallel and Distributed Computing

Conferences

• ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming (PPoPP)
• ACM Symposium on Parallel Algorithms and Architectures (SPAA)
• ACM/IEEE International Conference for High-performance Computing, Networking, Storage, and Analysis (SC)
• IEEE International Parallel and Distributed Processing Symposium (IPDPS)
• International Conference on Parallel Processing (ICPP)