What it is
LiteRT-LM is Google's C++ inference framework for running language models locally on Android, iOS, web, desktop, and devices like the Raspberry Pi, with GPU and NPU acceleration. It handles text, vision, and audio inputs, function calling, speculative decoding, and embedding models such as Gemma, Llama, Phi-4, and Qwen. App developers embed it through Python, Kotlin, Swift, JavaScript, and C++ APIs, or try models from the terminal with a CLI that can also serve an OpenAI-compatible endpoint.