# OpenAI Realtime Voice Agent (Browser WebRTC)

> Build a speech-to-speech voice agent in the browser with OpenAI's Realtime API, ephemeral client secrets, and the Agents SDK RealtimeSession.

Category: AI Agent Workflows  
Tags: voice, realtime, openai, webrtc, agents sdk  
Setup time (estimate): 45-90 minutes  
Estimated cost: OpenAI Realtime usage (audio and text tokens); see OpenAI's pricing page for gpt-realtime-2.1. Ephemeral client secrets are short-lived; no separate SDK fee for @openai/agents.  
Last verified: 2026-10-15

A browser voice agent that talks over WebRTC: your server mints a short-lived ephemeral key, the page connects a `RealtimeSession`, and the model handles audio turns, barge-in, and tools in one realtime session.

## Steps

### Step 1 — Install

In a browser app (Vite or Next.js), install the Agents SDK and Zod v4:

```bash
npm install @openai/agents zod
```

### Step 2 — Mint an ephemeral client secret on your server

Never put a project `sk-` key in the browser. Your backend calls `POST https://api.openai.com/v1/realtime/client_secrets` with a server API key:

```bash
curl -X POST https://api.openai.com/v1/realtime/client_secrets \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"session":{"type":"realtime","model":"gpt-realtime-2.1"}}'
```

Use the response top-level `value` field (prefix `ek_`) as the client secret. Mint a fresh one when needed; tokens are short-lived.

### Step 3 — Create the agent and session in the browser

```javascript
import { RealtimeAgent, RealtimeSession } from "@openai/agents/realtime";

const agent = new RealtimeAgent({
  name: "Assistant",
  instructions: "You are a helpful voice assistant.",
});

const session = new RealtimeSession(agent, {
  model: "gpt-realtime-2.1",
});

await session.connect({
  apiKey: "ek_...(ephemeral key from your server)",
});
```

### Step 4 — Run over HTTPS or localhost

Grant microphone permission, and talk. In the browser the SDK uses WebRTC and configures mic capture and playback for you. On Node it falls back to WebSocket.

### Step 5 — Optional tools and safety

Attach tools, handoffs, and guardrails to `RealtimeAgent` the same way as a text agent. If you identify end users, set `OpenAI-Safety-Identifier` on the server request that creates the client secret (hashed internal user ID).

## Notes

- Prefer the GA flow: no `OpenAI-Beta: realtime=v1` header; use `session.type` and `/v1/realtime/client_secrets`.
- Standalone package alternative: `@openai/agents-realtime` with the same `RealtimeAgent` / `RealtimeSession` imports from that package.
- Check current model IDs and prices on OpenAI's model and pricing pages before shipping; audio tokens dominate long calls.

## Sources (official docs, checked 2026-10-15)

- [Getting started with the Realtime API](https://developers.openai.com/api/docs/guides/realtime)
- [Voice Agents Quickstart (Agents SDK)](https://openai.github.io/openai-agents-js/guides/voice-agents/quickstart/)
- [GPT-Realtime-2.1 model](https://developers.openai.com/api/docs/models/gpt-realtime-2.1)
- [OpenAI pricing](https://developers.openai.com/api/docs/pricing)
- [npm @openai/agents](https://www.npmjs.com/package/@openai/agents)

## Failure modes

- Putting a long-lived sk- project key in the browser exposes it; mint ek_ ephemeral secrets on your server via POST /v1/realtime/client_secrets.
- Microphone permission denied or missing HTTPS/localhost blocks WebRTC capture.
- Stale Realtime beta clients still send OpenAI-Beta: realtime=v1 or old event shapes; use the GA client_secrets and session.type flow.
- Audio token cost rises quickly on long sessions; end idle sessions and check current Realtime pricing before going live.