What it is
AirLLM is a Python library that splits a model into layers on disk and streams them through the GPU one at a time, so models with tens or hundreds of billions of parameters can run inference in a few gigabytes of VRAM. It loads Hugging Face checkpoints through one AutoModel call, supports Llama, Qwen, DeepSeek, Mistral and others, and offers optional block-wise quantization for speed. Newer versions also allow adapter training on huge models with small cards. Hobbyists and researchers with consumer GPUs or Macs use it to try large open models they could not otherwise fit, trading speed for memory.