ggml

What it is

A tensor library written in plain C and C++ with no external dependencies, used as the compute core behind llama.cpp and whisper.cpp. Its kernels are hand-optimized with SIMD on ARM, x86, and RISC-V processors, it runs on CPU, GPU, NPU, and WebAssembly backends, and it supports quantization from 2 to 8 bits plus the GGUF format for storing models. Developers building inference engines or embedding models into native apps use it to run neural networks efficiently on ordinary hardware without allocating memory at runtime.