MiniMind

What it is

MiniMind is a from-scratch PyTorch implementation of a small LLM (around 64M parameters, plus MoE variants) with the full training pipeline: tokenizer training, pretraining, SFT, LoRA, DPO, PPO/GRPO reinforcement learning, tool use, agentic RL and distillation. Core algorithms are written without high-level wrappers, and the project ships datasets, released weights, an OpenAI-compatible server and a simple chat web UI. Learners and researchers work through it to understand how language models are built and to reproduce a full training run in a couple of hours on one GPU.