DeepSeek-V3
Open-weight 671B Mixture-of-Experts language model with 128K context
What it is
DeepSeek-V3 is a Mixture-of-Experts language model that has 671B parameters in total but activates only 37B for each token, and it ships as base and chat weights with a 128K context window. It uses Multi-head Latent Attention, auxiliary-loss-free load balancing, a multi-token prediction objective, and FP8 mixed-precision training. The repository includes inference code and pointers for running the weights locally through community engines and hardware vendors. Researchers and teams with large GPU clusters use it to run or build on a frontier-scale open model for chat, code, and reasoning tasks.