Skip to main content
Roboflow provides computer vision tools for object detection. The plugin offers both cloud-hosted inference (access to pre-trained models from Roboflow Universe) and local RF-DETR models.
Vision Agents requires a Stream account for real-time transport. Most providers offer free tiers to get started.

​
Installation

uv add "vision-agents[roboflow]"

​
Cloud Detection

Uses Roboflow’s hosted API with pre-trained models. 
from vision_agents.core import Agent, User
from vision_agents.plugins import roboflow, gemini, getstream

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You are a sports analyst.",
    llm=gemini.Realtime(fps=10),
    processors=[
        roboflow.RoboflowCloudDetectionProcessor(
            model_id="football-players-detection-3zvbc/20",
            classes=["player"],
            conf_threshold=0.5,
        )
    ],
)
Set ROBOFLOW_API_KEY in your environment or pass api_key directly.
NameTypeDefaultDescription
model_idstr—Roboflow Universe model ID
classesList[str]NoneClasses to detect (or all if None)
conf_thresholdfloat0.5Confidence threshold
fpsint5Frame processing rate
annotateboolTrueDraw bounding boxes

​
Local Detection

Runs RF-DETR models locally without API calls. 
processor = roboflow.RoboflowLocalDetectionProcessor(
    model_id="rfdetr-base",
    classes=["person"],
    conf_threshold=0.5,
)
NameTypeDefaultDescription
model_idstr"rfdetr-seg-preview"RF-DETR model ("rfdetr-nano", "rfdetr-base", "rfdetr-large")
classesList[str]NoneClasses to detect
conf_thresholdfloat0.5Confidence threshold

​
Cloud vs Local

CloudLocal
Use whenAccess to Roboflow Universe modelsHigher throughput, avoid rate limits
ProsThousands of pre-trained models, no GPU requiredNo API costs, lower latency, works offline
ConsRequires API key, potential rate limitsRequires local compute, RF-DETR models only

​
Next Steps

Build a Voice Agent

Get started with voice

Build a Video Agent

Add video processing