Horizontal Scaling - Vision Agents

By default, AgentLauncher tracks sessions in local memory. This works for single-node deployments, but breaks when you scale horizontally — each node only knows about its own sessions. The SessionRegistry with a RedisSessionKVStore solves this by sharing session state across all nodes via Redis. Any node can query or close sessions running on any other node, and sticky sessions are no longer required.

Why Multi-Node?

Scalability — Handle more concurrent voice sessions by distributing across nodes
Reliability — If one node goes down, new sessions route to healthy nodes
Session visibility — Any node can query or close sessions running on any other node

Installation

uv add "vision-agents[redis]"

This installs redis[hiredis] as an optional dependency.

Quick Start

from vision_agents.core import AgentLauncher, Runner
from vision_agents.core.agents.session_registry import (
    RedisSessionKVStore,
    SessionRegistry,
)

# 1. Create the Redis-backed store
store = RedisSessionKVStore(url="redis://localhost:6379")

# 2. Create the registry
registry = SessionRegistry(store=store)

# 3. Pass to AgentLauncher
runner = Runner(
    AgentLauncher(
        create_agent=create_agent,
        join_call=join_call,
        registry=registry,
    )
)
runner.cli()

Without a registry, AgentLauncher falls back to an in-memory store automatically. Existing single-node deployments continue to work with no changes.

How It Works

Registration — When start_session() is called, the session is registered in the store with a TTL (default 30s)
Heartbeat — The maintenance loop periodically refreshes the TTL for all active sessions on this node, keeping them alive in the store
Cross-node close — Calling the close endpoint on any node writes a close-request flag to the store. The node owning that session picks it up during the next maintenance cycle and shuts it down
Expiry — If a node crashes, its sessions’ TTLs expire naturally. Other nodes see those sessions disappear from the registry

The ttl value must be significantly higher than maintenance_interval to avoid sessions expiring between heartbeats. The default of 30s TTL with 5s maintenance interval provides a comfortable margin.

Architecture

The system has three layers:

SessionKVStore — Abstract key-value store with TTL support. Two built-in implementations:
- InMemorySessionKVStore — Used by default for single-node deployments
- RedisSessionKVStore — For multi-node production deployments
SessionRegistry — Facade that manages session lifecycle: registration, heartbeat refresh, cross-node close requests, and expiry detection
AgentLauncher — Accepts an optional registry parameter. When provided, the maintenance loop refreshes TTLs and processes close requests from other nodes

Configuration

SessionRegistry

Parameter	Type	Default	Description
`store`	`SessionKVStore`	`None`	Key-value store backend. `None` uses in-memory store
`node_id`	`str \| None`	`None`	Unique ID for this node. Auto-generated if `None`
`ttl`	`float`	`30.0`	Time-to-live in seconds for session keys

RedisSessionKVStore

Parameter	Type	Default	Description
`client`	`Redis`	`None`	Existing async Redis client instance
`url`	`str \| None`	`None`	Redis connection URL (used if `client` is `None`)
`key_prefix`	`str`	`"vision_agents:"`	Prefix for all keys in Redis

Either client or url must be provided. Pass client to reuse an existing Redis connection pool, or url for convenience.

InMemorySessionKVStore

Used automatically when no store is provided. Suitable for single-node deployments and development.

Parameter	Type	Default	Description
`cleanup_interval`	`float`	`60.0`	Interval in seconds between expired key cleanup

Custom Store Backend

The SessionKVStore is an abstract class with a simple interface. You can implement your own backend for any key-value store that supports TTL-based expiry (DynamoDB, Memcached, etcd, etc.). Subclass SessionKVStore and implement these abstract methods:

from vision_agents.core.agents.session_registry import SessionKVStore


class DynamoDBSessionKVStore(SessionKVStore):

    async def start(self) -> None:
        """Open connections. Called once when the registry starts."""
        ...

    async def close(self) -> None:
        """Close connections. Called once when the registry stops."""
        ...

    async def set(
        self, key: str, value: bytes, ttl: float, *, only_if_exists: bool = False
    ) -> None:
        """Store a value with a TTL in seconds.

        If only_if_exists is True, silently skip if the key doesn't exist.
        """
        ...

    async def mset(self, items: list[tuple[str, bytes, float]]) -> None:
        """Store multiple (key, value, ttl) tuples."""
        ...

    async def get(self, key: str) -> bytes | None:
        """Retrieve a value, or None if expired/missing."""
        ...

    async def mget(self, keys: list[str]) -> list[bytes | None]:
        """Retrieve multiple values. Return None for missing keys."""
        ...

    async def expire(self, *keys: str, ttl: float) -> None:
        """Refresh TTL on existing keys without changing values."""
        ...

    async def keys(self, prefix: str) -> list[str]:
        """Return all non-expired keys matching the prefix."""
        ...

    async def delete(self, keys: list[str]) -> None:
        """Delete one or more keys. Ignore missing keys."""
        ...

Then pass it to SessionRegistry:

store = DynamoDBSessionKVStore(table_name="sessions")
registry = SessionRegistry(store=store)

The store works with raw bytes — all serialization is handled by SessionRegistry. Your implementation only needs to store and retrieve byte values with TTL expiry.

Next Steps

Docker Deployment

Docker, Kubernetes, and scaling basics

Built-in HTTP Server

HTTP server API reference and configuration

Telemetry & Metrics

Monitor agent performance in production

Deploy Example

Complete Helm chart implementation

​Why Multi-Node?

​Installation

​Quick Start

​How It Works

​Architecture

​Configuration

​SessionRegistry

​RedisSessionKVStore

​InMemorySessionKVStore

​Custom Store Backend

​Next Steps

Docker Deployment

Built-in HTTP Server

Telemetry & Metrics

Deploy Example

Why Multi-Node?

Installation

Quick Start

How It Works

Architecture

Configuration

SessionRegistry

RedisSessionKVStore

InMemorySessionKVStore

Custom Store Backend

Next Steps