LocalAI

What it is

LocalAI is a server that runs open models on your own hardware and exposes them through APIs compatible with OpenAI, Anthropic and ElevenLabs. Inference engines such as llama.cpp, vLLM, whisper.cpp and stable-diffusion each come as their own backend, downloaded only once a model calls for it, and hardware support covers plain CPU as well as Apple Silicon, Vulkan, and GPUs from NVIDIA, AMD and Intel. It also includes multi-user auth with quotas, built-in agents with tool use, RAG and MCP, fine-tuning and quantization, and a terminal chat agent. Developers and teams use it to keep AI workloads and data on their own machines instead of calling a cloud API.