SIA
Self-improvement loop that rewrites an AI agent to score higher on a benchmark
Featured by Willet
What it is
SIA runs a meta agent that drafts a task-specific agent, executes it against a benchmark, then has a feedback agent read the logs and revise the agent's code (and optionally its weights) over successive generations. It ships with bundled tasks such as GPQA and LawBench, supports Claude or OpenHands-based agents across many model providers, and includes a web dashboard for inspecting each generation's code, scores, and trajectories. Researchers and ML engineers use it to automatically tune agents or models for a measurable task, with an optional Docker sandbox for running generated code.