What it is
xAI's Grok-1 base model weights, released under Apache 2.0, along with example JAX code that loads the checkpoint and samples text from it. The model uses eight experts with two active per token, 64 layers, and an 8,192-token context window. Researchers and engineers with multi-GPU machines can use it to study or build on a large open mixture-of-experts model. The included MoE layer is written for correctness rather than speed.