Discover
Every project in the index. Underrated first: good projects that haven’t been noticed yet.
52 projects
Remove filter #llm-inference- 15.5k
ggml
Dependency-free C/C++ tensor library for running ML models anywhere
- 16k
Outlines
Python library that constrains LLM generation to schemas and grammars
- 5.4k
NVIDIA Model Optimizer
Python library for quantizing, pruning, and distilling models for inference
- 19.6k
KTransformers
CPU-GPU hybrid inference and fine-tuning for large MoE language models
Jun Kim23k
- 24.1k
Paddle
Deep learning framework for training and deploying neural networks at scale
- 37.3k
MediaPipe
On-device ML framework and ready-made tasks for vision, audio, and text
- Tashfeen38.1k
FreeLLMAPI
Self-hosted gateway that pools free LLM API tiers behind one endpoint
Vincenzo Fornaro43k
- Ettore Di Giacinto49.8k
LocalAI
Self-hosted engine serving local LLMs, image, voice and video models
- 78.3k
Unsloth
Desktop and web app for running and fine-tuning local LLMs and diffusion models
- 132k
llama.cpp
C/C++ engine for running large language models on local hardware
- 168k
Hugging Face Transformers
Python library defining pretrained models for text, vision, and audio
184kOllama
Download and run open language models on your own computer with one command
- 201k
TensorFlow
Google's framework for building, training, and deploying ML models
- Salvatore Sanfilippo24k
DwarfStar
Native C inference engine for running a few large open models locally
- 25.1k
Google AI Edge Gallery
Mobile app for running open LLMs and multimodal models offline on your phone
- 26.9k
MiniCPM-V
Open multimodal models for images, video, documents and speech on-device
- 33.8k
tinygrad
Small, readable deep learning framework with its own kernel compiler
- Alex Jones38.2k
llmfit
Terminal tool that matches local LLMs to your CPU, RAM, and GPU
Gavin Li39.7k
- Andrej58.8k
nanochat
Minimal codebase for training a small chat LLM end to end on one GPU node
- 11.2k
PersonaPlex
Full-duplex speech-to-speech model with prompt-set roles and voices
52.2k