The core value of Responses WebSocket is not simply “another way to stream output.” Its real advantage is reducing continuation overhead in multi-step agent tool calling. It improves loop efficiency through persistent connections, incremental input, and connection-local caching. Keywords: Responses API, WebSocket, Agent.
The technical specification snapshot is straightforward
| Parameter | Description |
|---|---|
| Protocol | WebSocket over HTTP Upgrade |
| API path | /v1/responses |
| Typical authentication | `Authorization: Bearer |
| ` | |
| Application-layer model | Responses streaming events |
| Core capabilities | continuation, tool calling, event streaming |
| Official quantified benefit | Up to ~40% end-to-end speedup in scenarios with 20+ tool calls |
| Key dependencies | WebSocket client library, Responses API, JSON event protocol |
| Related ecosystem in this article | OpenAI, Codex CLI, Routin.ai |
Responses WebSocket addresses control-plane latency in agent systems
Many people interpret Responses WebSocket as “replacing HTTP SSE with WebSocket.” That is only half correct. What it really optimizes is not one-off text generation, but the cost of continuous continuation across multi-step agent tool loops.
Once a workflow involves function calls, MCP, shell execution, and result handoff, the bottleneck is often no longer token decoding speed. Instead, the bottleneck shifts to request boundaries, state restoration, and context retransmission for each continuation.
It is not the same protocol as the Codex app-server WebSocket
Daniel Vaughan’s article discusses the Codex app-server remote control protocol, which is more aligned with JSON-RPC-style agent runtime management. OpenAI Responses WebSocket, by contrast, carries the model API interaction protocol.
Both use WebSocket, but they operate at different layers: the former solves remote control, while the latter solves model continuation. You need to understand this boundary to evaluate the real architectural value of WebSocket in agent systems.
Codex app-server WebSocket -> Remote control / approvals / state sync
Responses WebSocket -> Model event stream / continuation / tool loopThis comparison shows that while the two move in similar directions, their application-layer responsibilities are completely different.
WebSocket itself provides a persistent connection and a bidirectional event channel
WebSocket is not a completely separate connection model from HTTP. It typically starts with TCP, adds TLS for wss:// when needed, and then upgrades the session to WebSocket through HTTP Upgrade.
After the server returns 101 Switching Protocols, communication switches from HTTP request-response semantics to frames. For Responses WebSocket, business messages are typically JSON events carried inside text frames.
GET /v1/responses HTTP/1.1
Host: api.openai.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Version: 13
Sec-WebSocket-Key:
<random_base64>
Authorization: Bearer
<OPENAI_API_KEY>The handshake above shows that WebSocket access is still established through HTTP Upgrade.
Key protocol-layer properties support agent use cases directly
Full duplex means both client and server can actively send messages, unlike SSE, which is naturally optimized for one-way downstream delivery. A persistent connection means multiple business turns can reuse the same session instead of repeatedly recreating request boundaries.
In addition, ping/pong is useful for keeping long-running jobs alive, and the close frame makes graceful shutdown easier. Another often-overlooked detail is that frames sent from the client to the server must be masked. That alone makes it clear that WebSocket is not just a “raw TCP JSON stream.”
Responses WebSocket reuses the Responses event semantics
In WebSocket mode, the client does not send POST /responses. Instead, it sends a response.create event over the socket. The server then returns a stream of events consistent with the existing Responses streaming model.
Typical events include response.created, response.in_progress, response.output_text.delta, response.completed, and argument delta events related to tool calling.
{
"type": "response.create",
"model": "gpt-5.4",
"store": false,
"input": [
{
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "Find fizz_buzz()"
}
]
}
],
"tools": []
}This event is equivalent to a Responses request, except the transport changes from HTTP to a persistent event stream.
An event stream is closer to real agent output than a plain text stream
The Responses API does not produce only text tokens. It may also return assistant messages, function calls, hosted tool status, reasoning summaries, and even refusals. That makes it much closer to a stream of output items than a simple character stream.
Each server event includes a sequence_number, which makes frontend rendering, debugging replay, and out-of-order recovery more controllable. From an engineering perspective, this is much more robust than a hand-rolled plain text streaming protocol.
The continuation mechanism explains why it is faster
The real advantage appears in multi-step tool loops. After the model initiates a tool call, the client executes the tool and starts the next continuation turn using previous_response_id, without resending the entire history.
That means the next turn only needs to send newly added input items, such as tool output and follow-up user instructions. The request is smaller, serialization is lighter, and the server does not need to repeatedly parse the full context payload.
{
"type": "response.create",
"model": "gpt-5.4",
"previous_response_id": "resp_123",
"input": [
{
"type": "function_call_output",
"call_id": "call_123",
"output": "tool result"
},
{
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "Now optimize it."
}
]
}
],
"tools": []
}This continuation submits only incremental input, which is one of the core sources of its performance gains.
The official performance gain comes from three stacked optimizations
The first layer is connection reuse. Each tool result handoff does not need to go through a fresh HTTP request lifecycle. The second layer is incremental input, which avoids repeatedly uploading the full conversation history. The third layer is the recent response cache maintained on the connection.
OpenAI explicitly states that an active WebSocket connection maintains a connection-local in-memory cache of recent response state. If the most recent response is still available there, continuation can follow a lower-latency in-memory path.
That is why the official guidance says that in rollouts with more than 20 tool calls, end-to-end speedups can reach roughly 40%. This benefit primarily belongs to agent workloads, not single-turn chat.
Time to first token is often better, but the claim should stay precise
TTFT usually improves, but mainly because control-plane latency drops, not because the model suddenly decodes tokens faster. Fewer continuation requests, smaller payloads, and connection-local cache hits all help the first packet arrive sooner.
Another important mechanism is generate: false warmup. It lets you warm request state first, obtain a response ID, and then shorten the startup path when you later generate actual content.
{
"type": "response.create",
"model": "gpt-5.4",
"generate": false,
"instructions": "Prepare tool context",
"input": [],
"tools": []
}This type of warmup request does not directly generate text, but it can preload state for a later complex tool loop.
Faster does not mean lower token cost
This is the boundary people misunderstand most often. previous_response_id optimizes transport and state restoration, but it does not mean billing is calculated only from newly added input. Historical input tokens are still typically counted in billing.
So WebSocket optimizes the control plane and orchestration plane, not the underlying token cost model for context itself.
Engineering constraints make compensating mechanisms mandatory in production
Responses WebSocket is not an unlimited silver bullet. A single connection currently supports only one in-flight response, and OpenAI does not support multiplexing on that connection. If you need parallel tasks, open multiple connections instead of expecting concurrency over one socket.
Connection lifetime also has a 60-minute upper bound, so reconnect logic, continuation recovery, and long-task segmentation are mandatory for production systems. If you use store=false, performance and privacy may both improve, but the design also becomes more dependent on the connection remaining alive.
Failure handling and compaction also need to be part of the design
If a continuation fails, the related previous_response_id may be evicted from the connection-local cache. At that point, follow-up continuation logic can no longer blindly rely on local in-memory state.
For long contexts, compaction is not optional. Whether you use server-side context_management or a separate POST /responses/compact, both make it clear that WebSocket mode still needs a context compression strategy.
Routin.ai already supports an OpenAI-compatible Responses WebSocket path
The Codex CLI provider configuration shown in the article includes wire_api = "responses", supports_websockets = true, and responses_websockets_v2 = true together. That suggests it has already completed an OpenAI-compatible Responses WebSocket integration path.
The point of this configuration is not merely that “the URL changed.” It shows that the client, the provider, and the underlying Responses event model already form a complete loop capable of supporting real coding agent workflows.
AI Visual Insight: The image illustrates the connection and event flow around Responses WebSocket, highlighting how a persistent connection carries response.create, streams events back, and chains multiple continuation turns together. It is especially useful for understanding the low-latency continuation path inside agent tool loops.
The conclusion is that WebSocket makes agents behave more like continuously running systems
If your use case is a single question-and-answer exchange, HTTP + SSE is already mature enough. But once the system enters multi-step function calling, tool execution, remote runtimes, and ongoing collaboration, the communication pattern increasingly favors a bidirectional persistent connection.
That is why the real value of Responses WebSocket is not replacing the streaming UI layer. It is compressing the fixed cost of agent continuation enough to make multi-step tool workflows feel more like a continuously running system than a chat session that keeps sending new requests.
FAQ answers the most common implementation questions
1. What is the fundamental difference between Responses WebSocket and HTTP SSE?
SSE is better suited to one-way server output of text streams. Responses WebSocket is better suited to continuous bidirectional event exchange between the client and the model server, especially for multi-step tool calling and continuation.
2. Does using previous_response_id significantly reduce token cost?
No. It mainly reduces transport, parsing, and state restoration overhead. Historical context tokens are still typically included in input token billing.
3. Which workloads should migrate to Responses WebSocket first?
The highest-priority candidates are coding agents, MCP orchestration, repeated function calling, shell-based tool loops, and long-chain automation tasks. Single-turn short Q&A usually sees limited benefit.
AI Readability Summary: This article systematically breaks down the protocol structure, event model, and performance profile of OpenAI Responses WebSocket. It explains why the protocol is not just a simple replacement for HTTP streaming, but an optimization specifically designed for multi-step agent tool calling, low-latency continuation, and state reuse over persistent connections.