Skip to content

About

A curated list of awesome high performance computing resources

Topics

Resources

Stars

1.4k stars

Watchers

42 watching

Forks

Latest commit

 

History

671 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 

Repository files navigation

A curated list of awesome high performance computing resources.

Table of Contents

General Info

Most Recent List of the Top500 Supercomputers

History

Trends

Software

Popular HPC Programming Libraries/APIs/Tools/Standards/Simulators

  • alpaka - The alpaka library is a header-only C++17 abstraction library for accelerator development
  • async-rdma - A framework for writing RDMA applications with high-level abstraction and asynchronous APIs
  • CANN - Compute Architecture for Neural Networks for Huawei Ascend GPUs
  • CAF - An Open Source Implementation of the Actor Model in C++
  • Chapel - A Programming Language for Productive Parallel Computing on Large-scale Systems
  • Charm++ - Parallel Programming with Migratable Objects
  • Cilk Plus - C/C++ Extension for Data and Task Parallelism
  • Codon - high-performance Python compiler that compiles Python code to native machine code without any runtime overhead
  • CUDA - High performance NVIDIA GPU acceleration
  • CUDA-oxide - custom rustc backend for compiling GPU kernels in pure Rust
  • dask - Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love
  • DeepSpeed - An easy-to-use deep learning optimization software suite that enables unprecedented scale and speed for Deep Learning Training and Inference
  • DeterminedAI - Distributed deep learning
  • Dispenso - Meta/facebook C++ Task Library
  • Faber - Experimental statically typed compute language for coding agents and human readers; tensor shapes live in the type, kernels run on CPU, CUDA and Metal device routes, and programs compile to Rust, C, Go, Swift and more
  • FastFlow - High-performance Parallel Patterns in C++
  • Galois - A C++ Library to Ease Parallel Programming with Irregular Parallelism
  • Halide - A language for fast, portable computation on images and tensors
  • Heteroflow - Concurrent CPU-GPU Task Programming using Modern C++
  • highway - Performance portable SIMD intrinsics
  • HIP - HIP is a C++ Runtime API and Kernel Language for AMD/Nvidia GPU
  • HPC-X - Nvidia implementation of MPI
  • HPX - A C++ Standard Library for Concurrency and Parallelism
  • Horovod - Distributed deep learning training framework for TensorFlow, Keras, PyTorch, and Apache MXNet
  • ISPC - An open-source compiler for high-performance SIMD programming on the CPU and GPU
  • Intel ISPC - SPMD compiler
  • Intel TBB - Threading Building Blocks
  • joblib - Data-flow programming for performance (python)
  • Kompute - The general purpose GPU compute framework for cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends)
  • Kokkos - A C++ Programming Model for Writing Performance Portable Applications on HPC platforms
  • Kubeflow MPI Operator - MPI Operator for Kubeflow
  • Legate - Nvidia replacement for numpy based on Legion
  • Legion - Distributed heterogeneous programming library
  • MAGMA - Next generation linear algebra (LA) GPU accelerated libraries
  • Merlin - A distributed task queuing system, designed to allow complex HPC workflows to scale to large numbers of simulations
  • Metal - Apple's GPU API
  • Microsoft MPI - Microsoft's implementation of MPI
  • MOGSLib - User defined schedulers
  • mpi4jax - Zero-copy mpi for jax arrays
  • mpi4py - Python bindings for MPI
  • MPI - OpenMPI implementation of the Message passing interface
  • MPI - MPICH implementation of the Message passing interface
  • MPI Standardization Forum - Forum for MPI standardization
  • MPAVICH - Implementation of MPI
  • NCCL - The NVIDIA Collective Communication Library for multi-GPU and multi-node communication
  • NVSHMEM - GPU-accelerated implementation of the OpenSHMEM programming model developed by NVIDIA
  • cuNumeric - GPU drop-in for numpy
  • stdpar - GPU accelerated C++ from NVIDIA
  • numba - A JIT compiler that translates a subset of Python into fast machine code
  • oneAPI - A unified, multiarchitecture, multi-vendor programming model
  • OpenACC - "OpenMP for GPUs"
  • OpenCilk - MIT continuation of Cilk Plus
  • OpenMP - Multi-platform Shared-memory Parallel Programming in C/C++ and Fortran
  • OpenSHMEM - OpenSHMEM is a one-sided, PGAS-based parallel programming model enabling direct remote memory access for high-performance computing
  • PVM - Parallel Virtual Machine: A predecessor to MPI for distributed computing
  • PMIX - Standard for process management
  • Pollux - Message Passing Cloud orchestrator
  • Pyfi - Distributed flow and computation system
  • Pyper - concurrent python made simple
  • RAJA - Architecture and programming model portability for HPC applications
  • RaftLib - A C++ Library for Enabling Stream and Dataflow Parallel Computation
  • ray - Scale AI and Python workloads from reinforcement learning to deep learning
  • ROCM - First open-source software development platform for HPC/Hyperscale-class GPU computing
  • RS MPI - Rust bindings for MPI
  • Scalix - Data parallel computing framework
  • Simgrid - Simulate cluster/HPC environments
  • SkelCL - A Skeleton Library for Heterogeneous Systems
  • STAPL - Standard Template Adaptive Parallel Programming Library in C++
  • STLab - High-level Constructs for Implementing Multicore Algorithms with Minimized Contention
  • SYCL - C++ Abstraction layer for heterogeneous devices
  • Taichi - Parallel programming language for high-performance numerical computations in Python
  • Taskflow - A Modern C++ Parallel Task Programming Library
  • The Open Community Runtime - Specification for Asynchronous Many Task systems
  • Transwarp - A Header-only C++ Library for Task Concurrency
  • Triton - Triton is a language and compiler for parallel programming
  • Tuplex - Blazing fast python data science
  • UCX - Optimized production proven-communication framework
  • Zluda - Run unmodified CUDA applications with near-native performance on Intel AMD GPUs.
  • HyperQueue - HyperQueue is a tool designed to simplify execution of large workflows (task graphs) on HPC clusters.

Cluster Hardware Discovery Tools

  • cpuid - A software instruction available on Intel, AMD, and other processors that can be used to determine processor type and features.
  • cpuid instruction note - A detailed note on the CPUID instruction used for processor identification.
  • cpufetch - A simple yet fancy CPU architecture fetching tool.
  • gpufetch - A tool similar to cpufetch, but for fetching GPU architecture.
  • intel cpuinfo - Intel tool providing information about the characteristics of Intel CPUs.
  • Likwid - Provides all information about the supercomputer/cluster.
  • LIKWID.jl - Julia wrapper for LIKWID.
  • openmpi hwloc - Portable Hardware Locality (hwloc) software project.
  • PRK - Parallel Research Kernels - A collection of kernels for parallel programming research.

Cluster Management/Tools/Schedulers/Stacks

  • ClusterVisor - Cluster management tool by Advanced Clustering.
  • BeeGFS - A parallel file system designed for performance-critical environments.
  • Bluebanquise - An open-source cluster management tool.
  • NVIDIA Base Command Manager (formerly Bright Cluster Manager) - Software for deploying and managing HPC and AI server clusters.
  • Ceph - An open-source distributed storage system.
  • DeepOps - Nvidia's GPU infrastructure and automation tools for Kubernetes and Slurm clusters.
  • E4S - The Extreme Scale HPC Scientific Stack - A collection of open-source software packages for HPC environments.
  • Easybuild - A package manager for HPC/supercomputers.
  • EESSI - A shared stack of scientific software installations.
  • Flux framework - A framework for high-performance computing clusters.
  • fpsync - A tool for fast parallel data transfer using fpart and rsync.
  • GPFS - A high-performance parallel file system developed by IBM.
  • Guix - A package manager for HPC/supercomputers.
  • Intel DAOS - A software-defined scale-out object store for HPC applications.
  • LSF - A batch system for HPC and distributed computing environments.
  • Lmod - A Lua-based module system for software environment management on HPC systems.
  • Lustre Parallel File System - A high-performance distributed filesystem for large-scale cluster computing.
  • moosefs - A fault-tolerant, highly available, distributed file system.
  • MrPackMod - TACC Alternative to Easybuild/Spack
  • NetApp - Intelligent data infrastructure for various workloads.
  • OKA - Analytics and reporting tool for HPC schedulers: helps administrators and users to understand how compute resources are used, and optimize their usage.
  • Open Cluster Scheduler - A scalable HPC/AI workload manager based on SGE.
  • OpenHPC - A community-led set of HPC components.
  • OpenOnDemand - A web portal for accessing supercomputing resources.
  • OpenPBS - A software for workload management and job scheduling.
  • OpenXdMod - A tool for managing high-performance computing resources.
  • RADIUSS - Rapid Application Development via an Institutional Universal Software Stack.
  • rocks - An open-source Linux cluster distribution.
  • Ruse - A tool for managing software environments in HPC clusters.
  • SGE - A resource management software for large clusters of computers.
  • Slurm - A cluster management and job scheduling system for Linux clusters.
  • Spectrum LSF - Workload management platform and job scheduler for distributed high performance computing (HPC)
  • Spack - A package manager for HPC/supercomputers.
  • Spinifex - Open-source, AWS-compatible private cloud for CPU and GPU clusters, with KVM instances, GPU passthrough, EBS, S3 and VPC on your own hardware.
  • sstack - A tool to install multiple software stacks such as Spack, EasyBuild, and Conda.
  • Starfish - Unstructured data management and metadata solution for files and objects.
  • Warewulf - An operating system provisioning system and cluster management tool.
  • Velda - A modern cluster management and job scheduler, with personalizable dev-containers and scale-to-cloud capabilities.
  • xCat - A distributed computing management and provisioning tool.
  • XDMoD - An open-source tool for managing high-performance computing resources.
  • Globus Connect - A fast data transfer tool between supercomputers.
  • Slurm Web - Open source web dashboard for Slurm HPC clusters.
  • s9s - TUI for SLURM cluster management
  • lazyslurm - A lazygit-style terminal UI for Slurm. Monitor jobs, tail logs, and inspect nodes and partitions.
  • slurm-why - Plain-English diagnoses for confusing Slurm job states (OOM kills, pending-reason codes, impossible requests) instead of decoding raw sacct/squeue/scontrol output by hand.

HPC-specific Operating Systems

  • Kitten - A lightweight kernel designed for high-performance computing. It focuses on providing low noise and predictable performance for HPC applications.
  • McKernel - A hybrid kernel that combines Linux and a lightweight kernel designed to provide high performance for HPC applications.
  • mOS - A specialized operating system for high-performance computing, designed to support large-scale, manycore processors.

Development/Workflow/Monitoring Tools for HPC

  • Apache Airflow - A platform to programmatically author, schedule, and monitor workflows.
  • Apptainer (formerly Singularity) - Container platform designed for scientific and high-performance computing (HPC) environments.
  • arbiter2 - Monitors and protects interactive nodes with cgroups.
  • Charliecloud - Lightweight container solution for high-performance computing (HPC).
  • Docker - A set of platform as a service products that use OS-level virtualization to deliver software in packages called containers.
  • genv - GPU Environment Management for managing and scheduling GPU resources.
  • Grafana - Open-source platform for monitoring and observability, visualizing metrics.
  • grpc - A high-performance, open-source universal RPC framework.
  • HPC Rocket - Allows submitting Slurm jobs in Continuous Integration (CI) pipelines.
  • HTCondor - An open-source high-throughput computing software framework.
  • Jacamar-ci - CI/CD tool designed for HPC and scientific computing workflows.
  • Kubernetes - An open-source system for automating deployment, scaling, and management of containerized applications.
  • nextflow - A workflow framework to deploy data-driven computational pipelines.
  • perun - Energy monitor for HPC systems, focusing on performance and energy efficiency.
  • Prefect - A workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine.
  • Prometheus - An open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.
  • redun - Workflow engine that emphasizes simplicity, reliability, and scalability.
  • remora - Tool for monitoring and reporting the performance of batch jobs on HPC systems.
  • ruptime - A utility for monitoring the status of computational jobs and systems.
  • server-spy - A monitoring tool for multi-run experiments on shared servers. It tracks congestion (PSI, scheduler wait) and shows how much specific runs got slowed down by other users' jobs, helping researchers identify skewed experiment comparisons.
  • slop - "top"-like command for slurm clusters
  • Slurmvision slurm dashboard - A dashboard for monitoring and managing Slurm jobs.
  • slurm docker cluster - A Slurm cluster implemented using Docker containers, for development and testing.
  • snakemake - A workflow management system that reduces the complexity of creating reproducible and scalable data analyses.
  • srunx - Python toolkit for managing SLURM jobs and workflows. Provides a CLI (sbatch / squeue / scancel parity), a FastAPI Web UI, an MCP server for natural-language control from Claude, YAML workflows with parameter sweeps and inter-job dependencies, GPU resource monitoring, SSH-based remote execution with rsync project sync, and Slack notifications.
  • Stui slurm dashboard for the terminal - A terminal-based UI for managing and monitoring Slurm clusters.
  • Vaex - A Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets.
  • HPC Client GUI - A cross-platform desktop client for SSH, SFTP, remote file management, Slurm job submission and monitoring, terminal workflows, and optional X11 forwarding. Runs locally and requires no server-side installation. Source-available for noncommercial use.

Debugging Tools for HPC

  • ddt - A powerful debugger designed for developers to solve complex problems on multi-threaded and multi-process environments in HPC.
  • marmot MPI checker - A tool for detecting and reporting issues in MPI (Message Passing Interface) applications.
  • python debugging tools - A collection of tools for debugging Python applications, including pdb and other utilities.
  • seer modern gui for gdb - A graphical user interface for GDB, aiming to improve the debugging experience with modern features and visuals.
  • Summary of C/C++ debugging tools - An overview of various debugging tools available for C/C++ applications, focusing on HPC environments.
  • totalview - A comprehensive source code analysis and debugging tool designed for complex software running on HPC systems, supporting a wide range of languages and architectures.

Performance/Benchmark Tools for HPC

  • demonspawn - A framework for automated execution of benchmarks and simulations, designed for HPC environments.
  • Google benchmark - A microbenchmark support library for C++ that tracks performance over time.
  • HPL benchmark - The High Performance Linpack Benchmark for measuring floating-point computing power of systems.
  • kerncraft - A tool for analytical modeling of loop performance and cache behavior on HPC systems.
  • NASA parallel benchmark suite - A set of benchmarks designed to evaluate the performance of parallel supercomputers.
  • papi - Provides standard APIs for accessing hardware performance counters available on modern microprocessors.
  • scalasca - A software tool that supports performance analysis of large-scale parallel applications.
  • scalene - A high-performance, high-precision CPU, GPU, and memory profiler for Python.
  • Summary of code performance analysis tools - An overview of tools for analyzing HPC application performance.
  • Summary of profiling tools - A comprehensive list of profiling tools for performance analysis in HPC.
  • tau - TAU (Tuning and Analysis Utilities) is a profiling and tracing toolkit for performance analysis of parallel programs.
  • The Bandwidth Benchmark - A tool for measuring memory bandwidth across various CPUs and systems.
  • vampir - A tool for detailed analysis of MPI program executions by visualizing their event traces.
  • bytehound memory profiler - A detailed memory profiler for tracking down memory issues and leaks.
  • Flamegraphs - Visualization tool for profiling software, allowing quick identification of performance bottlenecks.
  • flameox - Profiling and optimization toolkit for agents that captures and compares evidence from native services, PyTorch, GPU kernels, and inference workloads.
  • fio - Flexible I/O tester for benchmarking and stress/hardware verification.
  • IBM Spectrum Scale Key Performance Indicators (KPI) - Provides key performance indicators for IBM Spectrum Scale, aiding in performance tuning and monitoring.
  • Ior - A parallel file system I/O benchmarking tool used widely in HPC for testing storage systems.
  • ngstress - A versatile tool for stressing various subsystems of a computer to find hardware faults or to benchmark performance.
  • Pantheon - A diagnostics and stress-testing suite for NVIDIA and AMD GPUs that verifies its results to detect silent data corruption and reads the error counters of each card before and after a run.
  • Hotspot - The Linux perf GUI for in-depth performance analysis and visualization of software behavior.
  • HPC Challenge Benchmark Suite - benchmark suite that measures a range memory access patterns across CPU/GPU nodes.
  • mixbench - A benchmark suite designed to evaluate CPUs and GPUs across different compute and memory operations.
  • pmu-tools (toplev) - Performance monitoring tools for modern Intel CPUs, offering detailed insights into hardware and application performance.
  • SPEC CPU Benchmark - A benchmark suite designed to provide a comparative measure of compute-intensive performance across the widest practical range of hardware.
  • STREAM Memory Bandwidth Benchmark - Measures sustainable memory bandwidth and the corresponding computation rate for simple vector kernels.
  • Intel MPI benchmarks - A set of benchmarks designed to measure the performance and scalability of MPI implementations on Intel architectures.
  • Ohio state MPI benchmarks - A comprehensive suite of benchmarks for evaluating MPI performance across a variety of message passing patterns and communication protocols.
  • hpctoolkit - An integrated suite of tools for measurement and analysis of program performance on computers ranging from desktops to supercomputers.
  • core-to-core-latency - A diagnostic tool designed to measure and report the latency between CPU cores, aiding in the optimization of parallel computing tasks.
  • speedscope - An interactive, web-based viewer for performance profiles of software. It supports various formats and provides a flamegraph visualization to identify hot paths efficiently.
  • Differential Flamegraphs - A visualization technique developed by Brendan Gregg that highlights differences between performance profiles, making it easier to spot performance regressions or improvements.
  • Hyperfine - A command-line benchmarking tool that provides a simple and user-friendly means to compare the performance of commands, featuring statistical analysis across multiple runs.
  • Openfoam HPC benchmark - A benchmarking suite for evaluating the High Performance Computing capabilities of OpenFOAM, an open-source CFD software, under various computational loads.
  • OSU microbenchmarks - A collection of microbenchmarks designed to evaluate the performance of MPI implementations across various communication protocols and message sizes.
  • fio flexible I/O tester - A versatile tool for I/O workload simulation and benchmarking, capable of testing a wide array of storage and filesystem configurations.
  • vftrace - A tracing tool specifically designed for the NEC SX-Aurora TSUBASA Vector Engine, enabling detailed performance analysis of vectorized code.
  • tinymembench - A simple memory benchmark tool, focusing on benchmarking memory bandwidth and latency with minimal dependencies, suitable for various platforms.
  • Geekbench - Cross platform benchmarking tool
  • Empirical Roofline Tool (ERT) - Create empirical roofline plots, alternative to intel vtune for any machine
  • Roofline Visualizer for ERT - Visualizer for ERT
  • Caliper - A Performance Analysis Toolbox in a Library
  • KDiskMark - Benchmarking Tool For SSD/HDD Drives
  • OpenBenchmarking - Open benchmarks on a variety of algorithms and hardware
  • Phoronix Test Suite - Benchmarking suite for Linux
  • Palanteer Python/C++ Profiler - Profiler for both Python and C++
  • HECBioSim Benchmarks - The HECBioSim benchmark suite consists of a set of simple benchmarks for a number of popular Molecular Dynamics (MD) engines
  • JUPITER Benchmark Suite - The JUPITER Benchmark Suite incorporates 16 applications from various domains. It was designed for and used in the procurement of JUPITER, the first European exascale supercomputer.
  • OpenCL Cross-Hardware PhysX Benchmarks - Benchmark across all types of GPU, CPU
  • MLPerf benchmarks
  • Flopper.io - Comparison of GPUs
  • vhr-bench - A Bash script that benchmarks a Linux VPS or cloud node (sysbench CPU and memory, fio 4k random IOPS) in about a minute.

IO/Visualization Tools for HPC

  • ADIOS2 - The Adaptable IO System version 2, designed for flexible and efficient I/O for scientific data, supporting a wide range of HPC simulations.
  • Amira - A powerful, multifaceted 3D software platform for visualizing, manipulating, and understanding Life Science and bio-medical data coming from all types of sources.
  • hdf5 - The Hierarchical Data Format version 5 (HDF5), is an open source file format that supports large, complex, heterogeneous data.
  • paraview - An open-source, multi-platform data analysis and visualization application.
  • Scientific Visualization Wiki - A comprehensive guide to the field of scientific visualization, detailing techniques, tools, and applications.
  • the yt project - An open-source, Python-based package for analyzing and visualizing volumetric data.
  • vedo - A lightweight and powerful python module for scientific analysis and visualization of 3D objects and point clouds based on VTK.
  • visit - An Open Source, interactive, scalable, visualization, animation and analysis tool.
  • WebDataset - library for writing I/O pipelines for large datasets.

General Purpose Scientific Computing Libraries for HPC

Misc.

Wikis

Hardware

Vendors (Work in Progress)

Interconnects/Topology

CPU

GPU

TPU/Tensor Cores

Many integrated core processor (MIC)

Cloud

Vendors

Articles/Papers

Custom/FPGA/ASIC/APU

Certification

Student Opportunities / Workshops

Other/Wikis

People

Resources

Books/Manuals

Continuously updated (no single publication year):

Courses

Tutorials/Guides/Articles

General
Machine Learning Related

Review Papers/Articles

News

Podcasts

Video Presentations/Courses/Channels

Presentation Slides

Optimization Case Studies

Building Clusters/Virtual Clusters

Forums

Careers/Jobs

Membership Clubs

Blogs

Comics

Journals

Conferences

Communities/Chat Groups

Twitters

Consulting

Interview Preparation

Organizations

Summer schools

Interesting r/HPC posts

Misc. Wikis

Misc. Papers/Articles

Misc. Repos

Misc. Theses

Misc.

Games/Challenges

Other Curated Lists

Acknowledgements

This repo started from the great curated list https://github.com/taskflow/awesome-parallel-computing

About

A curated list of awesome high performance computing resources

Topics

Resources

Stars

1.4k stars

Watchers

42 watching

Forks

Releases

Packages

Contributors