Vision Agents
Python framework for real-time voice and video AI agents
What it is
Vision Agents is a Python framework for building agents that join live calls, watch video, and talk back with low latency over WebRTC. It chains vision models such as YOLO or Roboflow with realtime LLMs from OpenAI or Gemini, and plugs in speech-to-text, text-to-speech, turn detection, tool calling with MCP, RAG, and phone calls through Twilio or Telnyx. Developers use it to ship things like sports or workout coaching, video monitoring, and voice assistants, with Prometheus metrics and Kubernetes deployment included.