• Picks
  • Discover
  • Alternatives
  • About

© 2026 Willet

  • Newsletter
  • About
  • FAQ
  • Terms
  • Privacy & Cookies
Join the Discord (opens in a new tab)
Vincenzo Fornaro avatar

colibrì

CPU inference engine that streams MoE model experts from disk

  • AI models and runtimes
View on GitHub
Website

What it is

colibri is a dependency-free C inference engine that runs very large mixture-of-experts language models on ordinary computers by keeping the always-active weights in RAM and reading expert weights from disk as needed. It ships separate engines for model families such as GLM, DeepSeek, Kimi, Qwen and OLMoE, plus an image model, and can use CUDA or Vulkan GPUs when present. A setup script checks your RAM and disk, recommends a model that fits, downloads it with resume, and starts a local server with a browser dashboard and OpenAI- and Anthropic-compatible APIs. People who want to run frontier-scale open models locally without a big GPU use it in place of cloud APIs.

Images

Facts

Stars
43,005 GitHub stars.
43,005

Forks
4,823
Commits
3,738
Contributors
178
Language
C
License
Apache-2.0
Last commit
Oct 17, 2026
On GitHub since
July 2026
First release
July 2026
Platforms
web, macOS, Windows, Linux, CLI, self-hosted
Self-hostable
Yes
Alternative to
ChatGPT, Claude
Author
Vincenzo Fornaro
Tags
  • image-generation
  • llm-inference
  • local-ai
  • model-optimization

Last synced from GitHub on Oct 20, 2026

The 10 best overlooked open-source projects, every Monday.

Share and more

Comments

No comments yet. Start the conversation.

Similar projects

  • Salvatore Sanfilippo avatar

    DwarfStar

    Native C inference engine for running a few large open models locally

    24k
  • Microsoft avatar

    BitNet

    Inference engine for ternary 1.58-bit language models on CPU and GPU

    41.2k
  • Alex Jones avatar

    llmfit

    Terminal tool that matches local LLMs to your CPU, RAM, and GPU

    38.2k

What people are saying

Updated Oct 21, 2026
  • Hacker NewsShow HN: Getting GLM 5.2 running on my slow computerJul 9, 2026 · 937 points · 240 comments (opens Hacker News in a new tab)
  • Bluesky“colibrì — run AI models bigger than your machine”Sep 14, 2026 · 0 likes · 0 replies (opens Bluesky in a new tab)
  • Bluesky“A 744B MoE model running on consumer hardware with 25GB RAM, in pure C, zero dependencies. Colibri streams experts from disk on demand so you never need 756GB…”Jul 19, 2026 · 0 likes · 0 replies (opens Bluesky in a new tab)