> ## Documentation Index
> Fetch the complete documentation index at: https://visionagents.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Horizontal Scaling
> Scale Vision Agents across multiple servers with Redis-backed session management
By default, `AgentLauncher` tracks sessions in local memory. This works for single-node deployments, but breaks when you scale horizontally — each node only knows about its own sessions.
The `SessionRegistry` with a `RedisSessionKVStore` solves this by sharing session state across all nodes via Redis. Any node can query or close sessions running on any other node, and sticky sessions are no longer required.
## Why Multi-Node?
* **Scalability** — Handle more concurrent voice sessions by distributing across nodes
* **Reliability** — If one node goes down, new sessions route to healthy nodes
* **Session visibility** — Any node can query or close sessions running on any other node
## Installation
```bash theme={null}
uv add "vision-agents[redis]"
```
This installs `redis[hiredis]` as an optional dependency.
## Quick Start
```python theme={null}
from vision_agents.core import AgentLauncher, Runner
from vision_agents.core.agents.session_registry import (
RedisSessionKVStore,
SessionRegistry,
)
# 1. Create the Redis-backed store
store = RedisSessionKVStore(url="redis://localhost:6379")
# 2. Create the registry
registry = SessionRegistry(store=store)
# 3. Pass to AgentLauncher
runner = Runner(
AgentLauncher(
create_agent=create_agent,
join_call=join_call,
registry=registry,
)
)
runner.cli()
```
Without a `registry`, `AgentLauncher` falls back to an in-memory store automatically. Existing single-node deployments continue to work with no changes.
## How It Works
1. **Registration** — When `start_session()` is called, the session is registered in the store with a TTL (default 30s)
2. **Heartbeat** — The maintenance loop periodically refreshes the TTL for all active sessions on this node, keeping them alive in the store
3. **Cross-node close** — Calling the close endpoint on any node writes a close-request flag to the store. The node owning that session picks it up during the next maintenance cycle and shuts it down
4. **Expiry** — If a node crashes, its sessions' TTLs expire naturally. Other nodes see those sessions disappear from the registry
The `ttl` value must be significantly higher than `maintenance_interval` to avoid sessions expiring between heartbeats. The default of 30s TTL with 5s maintenance interval provides a comfortable margin.
## Architecture
The system has three layers:
1. **`SessionKVStore`** — Abstract key-value store with TTL support. Two built-in implementations:
* `InMemorySessionKVStore` — Used by default for single-node deployments
* `RedisSessionKVStore` — For multi-node production deployments
2. **`SessionRegistry`** — Facade that manages session lifecycle: registration, heartbeat refresh, cross-node close requests, and expiry detection
3. **`AgentLauncher`** — Accepts an optional `registry` parameter. When provided, the maintenance loop refreshes TTLs and processes close requests from other nodes
## Configuration
### SessionRegistry
| Parameter | Type | Default | Description |
| --------- | ---------------- | ------- | ---------------------------------------------------- |
| `store` | `SessionKVStore` | `None` | Key-value store backend. `None` uses in-memory store |
| `node_id` | `str \| None` | `None` | Unique ID for this node. Auto-generated if `None` |
| `ttl` | `float` | `30.0` | Time-to-live in seconds for session keys |
### RedisSessionKVStore
| Parameter | Type | Default | Description |
| ------------ | ------------- | ------------------ | ------------------------------------------------- |
| `client` | `Redis` | `None` | Existing async Redis client instance |
| `url` | `str \| None` | `None` | Redis connection URL (used if `client` is `None`) |
| `key_prefix` | `str` | `"vision_agents:"` | Prefix for all keys in Redis |
Either `client` or `url` must be provided. Pass `client` to reuse an existing Redis connection pool, or `url` for convenience.
### InMemorySessionKVStore
Used automatically when no store is provided. Suitable for single-node deployments and development.
| Parameter | Type | Default | Description |
| ------------------ | ------- | ------- | ----------------------------------------------- |
| `cleanup_interval` | `float` | `60.0` | Interval in seconds between expired key cleanup |
## Custom Store Backend
The `SessionKVStore` is an abstract class with a simple interface. You can implement your own backend for any key-value store that supports TTL-based expiry (DynamoDB, Memcached, etcd, etc.).
Subclass `SessionKVStore` and implement these abstract methods:
```python theme={null}
from vision_agents.core.agents.session_registry import SessionKVStore
class DynamoDBSessionKVStore(SessionKVStore):
async def start(self) -> None:
"""Open connections. Called once when the registry starts."""
...
async def close(self) -> None:
"""Close connections. Called once when the registry stops."""
...
async def set(
self, key: str, value: bytes, ttl: float, *, only_if_exists: bool = False
) -> None:
"""Store a value with a TTL in seconds.
If only_if_exists is True, silently skip if the key doesn't exist.
"""
...
async def mset(self, items: list[tuple[str, bytes, float]]) -> None:
"""Store multiple (key, value, ttl) tuples."""
...
async def get(self, key: str) -> bytes | None:
"""Retrieve a value, or None if expired/missing."""
...
async def mget(self, keys: list[str]) -> list[bytes | None]:
"""Retrieve multiple values. Return None for missing keys."""
...
async def expire(self, *keys: str, ttl: float) -> None:
"""Refresh TTL on existing keys without changing values."""
...
async def keys(self, prefix: str) -> list[str]:
"""Return all non-expired keys matching the prefix."""
...
async def delete(self, keys: list[str]) -> None:
"""Delete one or more keys. Ignore missing keys."""
...
```
Then pass it to `SessionRegistry`:
```python theme={null}
store = DynamoDBSessionKVStore(table_name="sessions")
registry = SessionRegistry(store=store)
```
The store works with raw bytes — all serialization is handled by `SessionRegistry`. Your implementation only needs to store and retrieve byte values with TTL expiry.
## Next Steps
Docker, Kubernetes, and scaling basics
HTTP server API reference and configuration
Monitor agent performance in production
Complete Helm chart implementation