> ## Documentation Index
> Fetch the complete documentation index at: https://visionagents.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Speech-to-Text and Text-to-Speech Class

LLMs not running using a `Realtime` model requires some help to convert the user's speech and LLM responses into something the user can speak to and hear. To achieve this, the `Agent` class exposes two parameters `tts` and `stt` allowing developers to pass in any text-to-speech and speech-to-text service they like. Using this method, the output voices can be configured, the transcription rate can be adjusted and more.

Internally, the Agent class handles the management between these services and things such as setting up the audio track for the STT providers as an example.

### STT (Speech-to-Text)

STT components convert audio input into text for processing by the LLM. All implementations follow a standardised interface with consistent event emission.

These components process real-time audio with `PcmData` objects from `getstream.video.rtc.track_util`, provide partial transcript support for responsive UI, and include comprehensive error handling and connection management. Multiple providers are supported including Deepgram, ElevenLabs, Fast Whisper, and others.

All STT providers must call `await stt.start()` before processing audio to initialize connections and resources.

Some STT providers include built-in turn detection (indicated by the `turn_detection` property). When this is the case, the Agent automatically skips any separately configured `TurnDetector` to avoid conflicts.

#### STT Methods

| Method                                 | Description                                                     |
| -------------------------------------- | --------------------------------------------------------------- |
| `start()`                              | Initialize connections and resources. Must be called before use |
| `process_audio(pcm_data, participant)` | Process an audio frame (\~20ms chunks)                          |
| `clear()`                              | Clear any pending audio or internal state                       |
| `close()`                              | Clean up resources                                              |

#### STT Events

| Event                       | Description                                                                        |
| --------------------------- | ---------------------------------------------------------------------------------- |
| `STTTranscriptEvent`        | Final transcript result                                                            |
| `STTPartialTranscriptEvent` | Interim transcript for real-time display                                           |
| `STTErrorEvent`             | Temporary, recoverable error                                                       |
| `STTConnectionEvent`        | Connection state change (DISCONNECTED, CONNECTING, CONNECTED, RECONNECTING, ERROR) |

### TTS (Text-to-Speech)

TTS components convert LLM responses into audio output. They handle audio synthesis and streaming to the output track.

These components provide streaming audio synthesis for low latency, multiple voice options and customisation, audio format standardisation using `PcmData` and `AudioFormat` from `getstream.video.rtc.track_util`, and support for providers like ElevenLabs, Cartesia, and others.

#### TTS Methods

| Method                                                   | Description                                                                               |
| -------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| `set_output_format(sample_rate, channels, audio_format)` | Configure output audio format. Audio is automatically resampled and re-channeled to match |
| `send(text, participant=None)`                           | Convert text to speech and emit audio events                                              |
| `stop_audio()`                                           | Clear the audio queue and stop current playback                                           |
| `close()`                                                | Clean up resources                                                                        |

#### TTS Events

| Event                       | Description                                                                                       |
| --------------------------- | ------------------------------------------------------------------------------------------------- |
| `TTSAudioEvent`             | Audio chunk ready for playback                                                                    |
| `TTSSynthesisStartEvent`    | Synthesis has begun for a text input                                                              |
| `TTSSynthesisCompleteEvent` | Synthesis finished (includes metrics like `synthesis_time_ms`, `chunk_count`, `real_time_factor`) |
| `TTSErrorEvent`             | Synthesis error (with recoverability flag)                                                        |
| `TTSConnectionEvent`        | Connection state change                                                                           |

#### Interruption support

The TTS base class exposes an `epoch` property and an `interrupt()` method for handling barge-in scenarios:

| Member        | Type    | Description                                                                                         |
| ------------- | ------- | --------------------------------------------------------------------------------------------------- |
| `epoch`       | `int`   | Monotonic counter that increments on each interruption. Used to identify stale audio events.        |
| `interrupt()` | `async` | Increments the epoch and stops the current audio synthesis. Stale events are automatically dropped. |

You do not need to call `interrupt()` manually — the Agent class invokes it when a `TurnStartedEvent` is received.