OpenAI’s gpt-realtime-2.1: Latency Is Not the Real Story

On July 6, 2026, OpenAI made gpt-realtime-2.1 and gpt-realtime-2.1-mini generally available in the Realtime API. No waitlist, no preview gate, just two new voice models for anyone with an API key. The number everyone is repeating is the latency figure: p95 latency down at least 25 percent across the Realtime voice models, driven mostly by improved caching. It is a real number and a good one. It is also not the story.

I have watched enough voice agent projects die to know that latency was rarely the killer. Demos with noticeable lag still wowed the room. What killed those projects in production was dumbness under pressure: the agent forgetting the account number from eight minutes earlier, mangling a medication name, plowing straight through an interruption, or losing the plot the moment a caller with an accent read out a confirmation code. The parts of this release that attack that class of failure are buried below the latency headline, and they deserve to be the headline.

Selectable reasoning effort is the real unlock

gpt-realtime-2.1 ships with five reasoning effort levels: minimal, low, medium, high, and xhigh, with low as the default. You set it per request. That per-request part matters more than the menu itself, because a live conversation is not one workload. Greeting a caller and confirming a shipping address is a minimal-effort job. Deciding whether a symptom description warrants escalation to a human nurse is not. Until now you picked one model and one behavior for the whole call. Now you can run the small talk fast and cheap, then crank the effort up for the three turns where deliberation actually pays, and drop it back down before the caller notices the pause.

This is the same routing logic that is reshaping text models. When I wrote about GPT-5.6’s Sol, Terra, and Luna tiers, my argument was that model choice is becoming a per-task routing decision rather than a brand loyalty decision. Realtime 2.1 pushes that decision inside a single phone call, which is exactly where voice needed it.

128K context turns demos into sessions

The second underrated change: the context window jumps from 32K to 128K tokens. Audio is token-hungry, and 32K was the reason long calls quietly degraded, with developers resorting to summarization hacks that threw away exactly the details a support call depends on. At 128K, everything said in the first minute of a forty-minute call is still sitting in context at the end. Coherence over a full session stops being an engineering trick and becomes a property of the model.

Stack the accuracy work on top of that and the healthcare angle stops sounding like marketing. OpenAI is claiming better retention of proper nouns, healthcare terms, and specialized vocabulary, improved alphanumeric recognition for the policy numbers and order IDs that agents flub constantly, plus better silence handling, noise handling, and interruption behavior. Transcription accuracy is up across Japanese, Korean, Chinese, Urdu, Vietnamese, and accented English, with long-form gains in Spanish, French, Italian, and Portuguese. That matters because a support line serves the callers it actually gets, not the callers in the demo video.

The tone adaptation is real too. The model speaks calmly while it works through an issue, shifts to something more empathetic when a caller is frustrated, and turns upbeat when confirming a success. I would rank this last among the reasons to upgrade. Do not buy this model for the vibes. Buy it for the memory and the judgment, and take the vibes as a bonus.

Then there is the money. The full-size gpt-realtime-2 runs about $32 per million audio input tokens, $0.40 when cached, and $64 per million audio output tokens, and audio tokens accumulate brutally fast at real call volumes. That is why gpt-realtime-2.1-mini is the quiet workhorse of this release: faster, cheaper, and priced the same as the earlier gpt-realtime-mini, so you get the 2.1 improvements without a new line on the budget. My default architecture now is mini for the bulk of traffic with escalation to the full model for hard turns, the same pattern I described with Claude Sonnet 5 as a default agentic model: a capable mid-tier handles most of the work and hands off the rest.

What I am not ready to believe yet

Three honest caveats before you rip out your current stack. First, xhigh reasoning will cost you latency, because deliberation is not free and physics does not care about your changelog. OpenAI has not published an effort-versus-latency curve, so measure it on your own turns before you promise anyone snappy responses. Second, that 25 percent p95 improvement leans on caching, and caching gains depend entirely on your traffic shape. If every call carries a personalized system prompt and fresh retrieval context, your cache hit rate drops, and so does your share of both the speedup and that $0.40 cached input rate. Third, gpt-realtime-2.1 is billed as OpenAI’s strongest realtime model for reasoning, tool use, and instruction following, but tool use inside a live audio stream is exactly the kind of claim that needs your eval harness rather than their benchmark table, especially before you put it in front of patients.

Here is what I would actually do this week. Point staging at gpt-realtime-2.1-mini with reasoning effort left at the default low. Log every turn where the agent fumbles, then test whether bumping effort to medium or high on those specific turns fixes them, and whether escalating to the full model fixes the rest. If your failure log is full of forgotten details and butchered ID numbers, this release was built for you. The 25 percent latency cut makes voice agents feel better. Selectable effort and 128K context are what make them deployable, and that is the difference between a demo and a product.