Switchyard
Per-request model routing for AI agents across cheap and capable LLMs
What it is
Switchyard is a Rust library with Python and TypeScript bindings that decides which model should handle each LLM request, mixing efficient and stronger models to trade off task success, cost, and latency. Routing strategies include classifier-based task judging, execution-stage routing driven by an agent's recent tool activity, and composites of the two. Teams running coding agents or LLM apps can embed it in their own harness, run the standalone local proxy that speaks OpenAI and Anthropic APIs, or plug it into gateways such as LiteLLM, NeMo Relay, or OpenRouter. Benchmark tooling helps compare a routed setup against a single-model baseline.