llama.cpp
C/C++ engine for running large language models on local hardware
What it is
llama.cpp runs LLM and vision-language model inference in plain C/C++ with no external dependencies, loading GGUF models straight from Hugging Face. It ships a command-line chat tool, a library API, and an OpenAI-compatible server with a built-in web UI. Quantization from 1.5 to 8 bits and backends for CUDA, Metal, Vulkan, HIP, SYCL and more let developers and hobbyists run models on laptops, GPUs, phones, or split across CPU and GPU.
