Stream Video RTC

Documentation Index

Fetch the complete documentation index at: https://visionagents.ai/llms.txt

Use this file to discover all available pages before exploring further.

​

Why Stream Video RTC

• Sub-500ms global latency. Agents connect through Stream’s edge network with PoPs worldwide — the same infrastructure that powers Stream Video for production telehealth, voice support, and live coaching apps.
• The default in every example. All the LLM, STT, TTS, vision, and realtime guides in these docs use getstream.Edge(). Swap providers freely; the edge stays the same.
• Audio + video + screen share. The plugin subscribes to audio, video, screen-share, and screen-share-audio tracks for every remote participant and re-publishes the agent’s own audio and video.
• Chat-backed conversation history. StreamConversation mirrors the message history to a Stream Chat channel attached to the call, with markdown-aware chunking and ephemeral updates while the LLM is still generating — so your frontend can render transcripts and tool output in real time.
• Custom events to every participant. Push arbitrary JSON to clients via send_custom_event(...) (payload capped at 5 KB by the platform) — useful for surfacing tool calls, UI hints, or telemetry.
• Built-in demo helper. open_demo(call) provisions a guest user, joins them to the chat channel, mints a token, and opens Stream’s hosted demo UI in your browser. Handy before you wire up a real client.
• Rich event surface. The plugin re-exports Stream’s call.* events — recording started/stopped, transcription ready, closed captions, HLS/RTMP broadcasting state, moderation actions, member updates, and more — so you can react to platform state from the agent’s event bus.
• First-class frontend SDKs. Web, React, React Native, iOS, Android, Flutter, and Unity clients join the same call as your server-side agent.
• Generous free tier. See Stream’s pricing for details.

​

Installation

uv add "vision-agents[getstream]"

​

Quick Start

from dotenv import load_dotenv

from vision_agents.core import Agent, AgentLauncher, User, Runner
from vision_agents.plugins import getstream, gemini

load_dotenv()


async def create_agent(**kwargs) -> Agent:
    return Agent(
        edge=getstream.Edge(),
        agent_user=User(name="Assistant", id="agent"),
        instructions="You're a helpful voice assistant. Be concise.",
        llm=gemini.Realtime(),
    )


async def join_call(agent: Agent, call_type: str, call_id: str, **kwargs) -> None:
    call = await agent.create_call(call_type, call_id)
    async with agent.join(call):
        await agent.simple_response("Greet the user")
        await agent.finish()


if __name__ == "__main__":
    Runner(AgentLauncher(create_agent=create_agent, join_call=join_call)).cli()

​

Environment Variables

​

Conversation Persistence

• Markdown-aware chunking — long messages are split into ~1000-character pieces, preserving code-block boundaries so frontend renderers don’t break mid-fence.
• Streaming updates — chunks are sent as ephemeral messages while the LLM is still generating, then finalized when generation completes. Clients see partial text update in real time.
• Bidirectional history — anything posted to the channel from a client SDK is available to the agent for memory or RAG.

​

Custom Events

await agent.edge.send_custom_event({
    "type": "tool_result",
    "tool": "search_orders",
    "result": {"order_id": "1234", "status": "shipped"},
})

​

Opening a Demo

async with agent.join(call):
    await agent.edge.open_demo(call)
    await agent.finish()

​

Platform Events

from vision_agents.plugins.getstream import (
    CallRecordingStartedEvent,
    CallTranscriptionReadyEvent,
    CallSessionParticipantJoinedEvent,
)

@agent.events.subscribe
async def on_recording(event: CallRecordingStartedEvent):
    print("Recording started:", event)

• Participants & members — CallSessionParticipantJoined/Left, CallMemberAdded/Removed/Updated, CallSessionStarted/Ended.
• Recording — CallRecordingStarted/Stopped/Ready/Failed, CallFrameRecordingStarted/Stopped/FrameReady/Failed.
• Transcription & captions — CallTranscriptionStarted/Stopped/Ready/Failed, CallClosedCaptionsStarted/Stopped/Failed, ClosedCaptionEvent.
• Broadcasting — CallHLSBroadcastingStarted/Stopped/Failed, CallRtmpBroadcastStarted/Stopped/Failed.
• Lifecycle — CallCreated/Updated/Deleted/Ended, CallRing/Accepted/Rejected/Missed/Notification/Reaction.
• Moderation & permissions — CallModerationBlur/Warning, BlockedUser/UnblockedUser/KickedUser, PermissionRequest, UpdatedCallPermissions, CallUserMuted.
• Telemetry — CallStatsReportReady, CallUserFeedbackSubmitted.

​

Frontend SDKs

​

Next Steps