nanochat
Minimal codebase for training a small chat LLM end to end on one GPU node
What it is
A compact, hackable PyTorch project that walks through every stage of building a chat model: tokenizer training, pretraining, finetuning, evaluation, and inference with a simple CLI or web chat. A single depth setting scales the model, with other hyperparameters derived automatically, and a reference script trains a GPT-2 grade model in about two hours on an 8xH100 node. Researchers and learners use it to study LLM training, run scaling experiments, and compete on a time-to-GPT-2 leaderboard.