Overview
LiveKit Agents supports an agent server architecture for managing multiple concurrent agent sessions and programmatic participants. Use dispatch, job execution, and configuration options to scale your agents horizontally and manage their lifecycles.
Programmatic participants
The Agents framework isn't limited to AI agents. You can use it to deploy any code that needs to process realtime media and data streams as a programmatic participant. A programmatic participant is any code that joins a LiveKit room as a participant—this includes AI agents, media processors, or custom logic that processes realtime streams.
Some examples of what these participants can do include:
- Process audio streams: Analyze audio for patterns, quality metrics, or content detection.
- Handle video processing: Apply computer vision, video effects, or content moderation.
- Manage data flows: Aggregate, transform, or route realtime data between participants.
- Provide services: Act as bridges to external APIs, databases, or other systems.
The framework provides the same production-ready infrastructure for all types of programmatic participants, including automatic scaling and load balancing. You can use the entrypoint function without creating an AgentSession to build programmatic participants that are automatically dispatched to rooms.
Processing raw media tracks
Learn how to process raw audio and video tracks in your programmatic participants.
Agent server components
Use core components to manage agent servers, including agent dispatch, job execution, and configuration.
| Component | Description | Use cases |
|---|---|---|
| Agent dispatch | Assign agents to rooms automatically or explicitly, with load balancing and high concurrency support. | Automatic agent assignment, explicit dispatch control, and custom dispatch logic. |
| Job lifecycle | Manage the entrypoint function, job execution, and session cleanup for each agent instance. | Entrypoint configuration, session management, and graceful shutdown. |
| Server options | Configure permissions, dispatch rules, prewarm functions, and server behavior. | Permission management, load balancing configuration, and server initialization. |
In this section
Read more about each component.