What it is
bitnet.cpp is Microsoft's inference framework for 1-bit (ternary) LLMs such as BitNet b1.58, with optimized kernels for x86 and ARM CPUs plus a GPU kernel. It runs these quantized models with large speedups and lower energy use than full-precision inference, and supports BitNet embedding models through I2_S conversion. Developers and researchers use it to run language and embedding models locally on ordinary hardware, including large models on a single CPU.