Discover

Every project in the index. Underrated first: good projects that haven’t been noticed yet.

26 projects

#llm-inference Python Clear filters

TokenSpeed
LLM serving engine tuned for agentic workloads on datacenter GPUs
AI models and runtimes Servers and infrastructure
2.7k
forge
Reliability layer that makes local LLM tool calls succeed more often
Agent frameworks and tools AI models and runtimes
Antoine2.3k
SIE
Self-hosted inference server running many open models behind one API
AI models and runtimes Agent frameworks and tools Servers and infrastructure
3.4k
OpenMed
Clinical entity extraction and PII de-identification on local hardware
AI models and runtimes Health and fitness Privacy and security
Maziyar Panahi5.7k
RLM
Inference library letting language models recursively call themselves via a REPL
Agent frameworks and tools AI models and runtimes
az5.7k
DFlash
Block diffusion draft models for faster LLM speculative decoding
AI models and runtimes
6.2k
MLX-VLM
Run and fine-tune vision language models locally on Apple silicon
AI models and runtimes
Prince Canuma5.6k
vllm-omni
Serving engine for multimodal, diffusion, speech, and action models
AI models and runtimes AI media
7.3k
ODS
Installer that sets up a complete private local AI stack on your own machine
AI models and runtimes Self-hosted alternatives AI apps
7.6k
whichllm
Command-line tool that ranks local LLMs that will run on your hardware
AI models and runtimes Developer tools
andy6.8k
TabPFN
Pretrained transformer for classification and regression on tables
AI models and runtimes Data and spreadsheets
8.2k
LMCache
KV cache layer that stores and reuses LLM attention state across engines
AI models and runtimes Servers and infrastructure
12.1k
Speech To Speech
Local voice agent pipeline chaining VAD, speech recognition, LLM and TTS
AI media Agent frameworks and tools AI models and runtimes
13.5k
Outlines
Python library that constrains LLM generation to schemas and grammars
Agent frameworks and tools AI models and runtimes
16k
NVIDIA Model Optimizer
Python library for quantizing, pruning, and distilling models for inference
AI models and runtimes
5.4k
KTransformers
CPU-GPU hybrid inference and fine-tuning for large MoE language models
AI models and runtimes
19.6k
oMLX
Local LLM server for Apple Silicon Macs with SSD-backed KV caching
AI models and runtimes
Jun Kim23k
Unsloth
Desktop and web app for running and fine-tuning local LLMs and diffusion models
AI models and runtimes AI apps
78.3k
Hugging Face Transformers
Python library defining pretrained models for text, vision, and audio
AI models and runtimes Languages and frameworks
168k
MiniCPM-V
Open multimodal models for images, video, documents and speech on-device
AI models and runtimes
26.9k
tinygrad
Small, readable deep learning framework with its own kernel compiler
AI models and runtimes Languages and frameworks
33.8k
nanochat
Minimal codebase for training a small chat LLM end to end on one GPU node
AI models and runtimes Education
Andrej58.8k
PersonaPlex
Full-duplex speech-to-speech model with prompt-set roles and voices
AI models and runtimes AI media
11.2k
Grok-1
314B-parameter mixture-of-experts language model with JAX loading code
AI models and runtimes
52.2k