A fast communication-overlapping library for tensor/expert parallelism on GPUs.

Our first fully AI generated deep learning system

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

Examples of CUDA implementations by Cutlass CuTe

CUTLASS and CuTe Examples

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

This is my 🔥 100 Days of GPU — a wild, hands-on journey through CUDA/CUTLASS kernels, Triton spells, and PTX sorcery.

GEMM and Winograd based convolutions using CUTLASS

study of cutlass

Multiple GEMM operators are constructed with cutlass to support LLM inference.

A cutlass cute implementation of headdim-64 flashattentionv2 TensorRT plugin for LightGlue. Run on Jetson Orin NX 8GB with TensorRT 8.5.2.

CUDA编程练习项目-Hands-on CUDA kernels and performance optimization, covering GEMM, FlashAttention, Tensor Cores, CUTLASS, quantization, KV cache, NCCL, and profiling.

Kernel Library Wheel for SGLang

Reading notes on the open source code of AI infrastructure (sglang, llm, cutlass, hpc, etc.)

DeepGEMMPerTensor: clean and efficient FP8 GEMM per tensor kernels without scales

NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process communication and coordination overheads by allowing programmers to perform one-sided communication from within CUDA kernels and on CUDA streams.

pytorch implements block sparse

Docs

Lightweight and production level C++ Open source Library