Discover

Every project in the index. Underrated first: good projects that haven’t been noticed yet.

52 projects

#llm-inference

Karukan
Japanese input method with neural kana-kanji conversion
System utilities AI models and runtimes
Hitoshi Togasaki740
FlashKDA
CUDA kernels for Kimi Delta Attention on Hopper-class GPUs
AI models and runtimes
1.3k
TokenSpeed
LLM serving engine tuned for agentic workloads on datacenter GPUs
AI models and runtimes Servers and infrastructure
2.7k
forge
Reliability layer that makes local LLM tool calls succeed more often
Agent frameworks and tools AI models and runtimes
Antoine2.3k
SIE
Self-hosted inference server running many open models behind one API
AI models and runtimes Agent frameworks and tools Servers and infrastructure
3.4k
Bonsai
Setup scripts to run Bonsai 2 low-bit models locally with chat UI
AI models and runtimes
3.4k
LiteRT
On-device runtime for running ML and generative AI models on edge hardware
AI models and runtimes
3.5k
Weave Router
Prompt-level model router with an OpenAI-compatible endpoint
AI models and runtimes AI coding agents
5.6k
OpenMed
Clinical entity extraction and PII de-identification on local hardware
AI models and runtimes Health and fitness Privacy and security
Maziyar Panahi5.7k
RLM
Inference library letting language models recursively call themselves via a REPL
Agent frameworks and tools AI models and runtimes
az5.7k
DFlash
Block diffusion draft models for faster LLM speculative decoding
AI models and runtimes
6.2k
LiteRT-LM
On-device LLM runtime for phones, desktops, browsers, and IoT boards
AI models and runtimes
6.7k
MLX-VLM
Run and fine-tune vision language models locally on Apple silicon
AI models and runtimes
Prince Canuma5.6k
Magnitude
Local LLM engine that auto-tunes GPU kernels for each model and chip
AI models and runtimes
7.2k
vllm-omni
Serving engine for multimodal, diffusion, speech, and action models
AI models and runtimes AI media
7.3k
GLM-5
Open-weight mixture-of-experts LLMs for coding and long agent tasks
AI models and runtimes
7.4k
ODS
Installer that sets up a complete private local AI stack on your own machine
AI models and runtimes Self-hosted alternatives AI apps
7.6k
whichllm
Command-line tool that ranks local LLMs that will run on your hardware
AI models and runtimes Developer tools
andy6.8k
TabPFN
Pretrained transformer for classification and regression on tables
AI models and runtimes Data and spreadsheets
8.2k
DeepGEMM
CUDA tensor core kernels for FP8, FP4 and BF16 GEMMs and MoE layers
AI models and runtimes
9k
DeepEP
GPU communication kernels for expert-parallel MoE training and inference
AI models and runtimes
10.3k
LMCache
KV cache layer that stores and reuses LLM attention state across engines
AI models and runtimes Servers and infrastructure
12.1k
FlashMLA
Optimized attention kernels for DeepSeek model inference on GPUs and NPUs
AI models and runtimes
13.1k
Speech To Speech
Local voice agent pipeline chaining VAD, speech recognition, LLM and TTS
AI media Agent frameworks and tools AI models and runtimes
13.5k