Speech To Text (STT)

Speech-to-Text (STT) converts spoken audio into text. In a custom voice pipeline, STT sits between the call and the LLM: user speaks → STT transcribes → turn signal → LLM → TTS. See Voice Agents for the full wiring guide.

Quick Start

uv add "vision-agents[deepgram,inworld,getstream]"

Add keys to your .env:

STREAM_API_KEY=...
STREAM_API_SECRET=...
DEEPGRAM_API_KEY=...

from dotenv import load_dotenv

from vision_agents.core import Agent, User
from vision_agents.plugins import deepgram, gemini, getstream, inworld

load_dotenv()

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You are a helpful voice assistant.",
    llm=gemini.LLM(),
    stt=deepgram.STT(eager_turn_detection=True),
    tts=inworld.TTS(),
)

The SDK captures audio from your Stream call and feeds it to the STT provider in real time. As people speak, you receive live transcripts.

Providers

Plugin	Turn detection	Latency model	Integration
Deepgram	Yes (eager optional)	Streaming	Deepgram
ElevenLabs	Yes (VAD commit)	Streaming	ElevenLabs
Cartesia	Yes (eager default)	Streaming	Cartesia
AssemblyAI	Yes	Streaming	AssemblyAI
Sarvam	Yes	Streaming	Sarvam
Fast-Whisper	No	Local batch (~2s buffer)	Fast-Whisper
Wizper	No	Cloud (Fal.ai)	Wizper
Fish	No	Batch	Fish
Mistral	No	Streaming	Mistral

Fast-Whisper runs locally on your machine. Wizper uses Fal.ai cloud — it is not a local model.

STT plugins without built-in turn detection need an external turn detection plugin, or the agent falls back to using the final transcript as the end-of-turn signal.

Transcripts

STT providers emit partial, replacement, and final transcripts. Subscribe to agent events to handle them in your application:

from vision_agents.core.agents.events import UserTranscriptEvent

@agent.events.subscribe
async def on_transcript(event: UserTranscriptEvent):
    print(event.text)

Next Steps

Voice Agents

Wire STT into a full voice pipeline

Turn Detection

When the agent starts and stops listening

Text To Speech (TTS)Speech To Speech (STS)

⌘I

Getting Started

AI Technologies

Core Reference

Reference

Quick Start

Providers

Transcripts

Next Steps

Voice Agents

Turn Detection

​Quick Start

​Providers

​Transcripts

​Next Steps

Voice Agents

Turn Detection

Quick Start

Providers

Transcripts

Next Steps