DeepEP
GPU communication kernels for expert-parallel MoE training and inference
What it is
DeepEP is a CUDA library from DeepSeek whose all-to-all kernels send tokens to experts and gather their outputs in mixture-of-experts models, with support for FP8 dispatch and NVLink exchange of weights and gradients for redundant experts. Kernels are compiled at runtime and run over NVLink within a node and RDMA across nodes, with experimental primitives for pipeline, context, and data parallelism. ML infrastructure engineers import it from PyTorch to speed up large-scale MoE training and serving on Hopper-class GPU clusters.