NVIDIA Model Optimizer
Python library for quantizing, pruning, and distilling models for inference
What it is
NVIDIA's model compression library takes Hugging Face, PyTorch, or ONNX models and applies quantization (FP8, NVFP4, INT4 and others), pruning, distillation, sparsity, neural architecture search, and speculative decoding. It exports optimized checkpoints that load into TensorRT-LLM, TensorRT, vLLM, and SGLang. ML engineers use it to shrink large language and diffusion models and raise serving throughput while recovering accuracy through quantization-aware training or distillation.