1# Invocations WebSocket Protocol Guide
2 
3The `invocations_ws` protocol is a **duplex WebSocket pass-through**. After the platform authenticates the upgrade request and routes it to your container, every frame in both directions is forwarded as-is. The agent developer defines the wire format, the framing model, and the streaming semantics. Unlike `responses` (OpenAI-compatible, platform-managed history) and `invocations` (single HTTP request/response, bytes in / bytes out), `invocations_ws` is a long-lived bidirectional channel under full container control.
4 
5## Input/Output Contract
6 
7| Aspect | `responses` | `invocations` | `invocations_ws` |
8|--------|-------------|---------------|------------------|
9| **Transport** | HTTPS request/response | HTTPS request/response | WebSocket (`wss://`) |
10| **Lifetime** | Per request | Per request | Long-lived duplex connection |
11| **Input** | Natural language `inputText` | Raw HTTP request body | Sequence of WS frames in either direction |
12| **Output** | Structured OpenAI JSON | Raw response bytes | Sequence of WS frames in either direction |
13| **Framing** | n/a (single body) | n/a (single body) | Developer-defined: binary (PCM, protobuf), text (JSON), or mixed |
14| **Streaming** | `stream: true` (SSE) | Agent-controlled (SSE-over-HTTP, etc.) | Native — duplex by definition |
15| **History** | Platform via `conversationId` | Agent-managed | Agent-managed; keyed by `agent_session_id` |
16 
17## URL and Headers
18 
19```
20wss://{account}.services.ai.azure.com
21   /api/projects/agents/endpoint/protocols/invocations_ws
22   ?project_name={project}
23   &agent_name={agentName}
24   &agent_session_id={sessionId}
25   &foundry_features=HostedAgents=V1Preview
26```
27 
28| Query parameter | Required | Notes |
29|-----------------|----------|-------|
30| `project_name` | ✅ | Foundry project name (the segment after `/api/projects/` in the project endpoint) |
31| `agent_name` | ✅ | Hosted agent name as declared in `agent.yaml` |
32| `agent_session_id` | ❌ | Per-connection identifier — see [Session Management](../../invoke/references/session-management.md). If omitted, the platform (or the container) generates a random id |
33| `foundry_features` | ✅ (preview) | Must be `HostedAgents=V1Preview` while the protocol is in preview. May alternatively be sent as the `Foundry-Features` request header. |
34 
35| Header | Required | Notes |
36|--------|----------|-------|
37| `Authorization: Bearer <token>` | ✅ | Entra token for audience `https://ai.azure.com` (scope `https://ai.azure.com/.default`) — `az account get-access-token --resource https://ai.azure.com`. Validated by APIM and the Agents service; the container does **not** see this header. |
38| `Foundry-Features: HostedAgents=V1Preview` | ✅ (preview) | Required unless the equivalent `foundry_features` query parameter is set. |
39 
40The container receives the upgrade on path `/invocations_ws`. Inside the container, read the session id from the `FOUNDRY_AGENT_SESSION_ID` environment variable (set by `azure-ai-agentserver-invocations`), or fall back to the `agent_session_id` query string.
41 
42> ⚠️ **Browsers cannot set the `Authorization` header on a `WebSocket`.** Browser clients must connect through a thin server-side proxy that adds the header before forwarding. This is a browser API limitation, not a Foundry requirement.
43 
44## Pass-Through Semantics
45 
46The platform is a transparent relay:
47 
48- **No schema validation.** Binary opcodes, text JSON, protobuf, raw PCM — anything ends up at the container untouched.
49- **No transcoding.** Sample rate, codec, byte order are entirely between caller and container.
50- **No history.** Nothing is persisted by Foundry between connections. Use the container filesystem or an external store, keyed by `agent_session_id`, if you need continuity.
51- **No platform-managed turn taking.** There is no concept of "request" vs "response" — both sides may send frames at any time. Implement your own request/reply correlation if you need it (e.g. include an `id` field in each JSON frame).
52 
53## Common Framing Patterns
54 
55These are protocols developers build **on top of** the raw WebSocket. The platform does not require, parse, or validate any of them; they are listed for orientation only.
56 
57| Pattern | Typical use | Notes |
58|---------|-------------|-------|
59| **Raw binary media frames** | Voice agents (PCM, Opus) | Binary opcode; agree on sample rate, channels, bit depth out-of-band |
60| **Length-prefixed protobuf** | Real-time pipeline frameworks | Each WS frame is one serialized message; control + audio multiplexed |
61| **JSON control + binary media** | Mixed signaling | Text frames carry control (e.g. start/stop, RTVI events), binary frames carry media |
62| **Pure JSON signaling** | Out-of-band media transports (WebRTC offer/answer/ICE, SFU join tokens) | One JSON object per frame; FIFO request/reply if the protocol is purely turn-based |
63| **SSE-style event stream** | One-way server push of events | Text frames; the WS is effectively used as a richer SSE |
64 
65## Discovering the Expected Wire Format
66 
67> ⚠️ **Do not guess.** The platform exposes no OpenAPI / AsyncAPI surface for `invocations_ws` agents. The contract lives in the container code.
68 
69### 1. Inspect the WebSocket Handler
70 
71Look at the function decorated with `@app.ws_handler` on an `InvocationAgentServerHost` (the `azure-ai-agentserver-invocations` SDK). The handler determines:
72 
73- Whether frames are binary, text, or mixed
74- The expected first frame (handshake, capabilities, auth challenge)
75- The control vocabulary (start, stop, mute, hangup, etc.)
76- The response cadence (turn-based vs free-running)
77 
78### 2. Ask the User or Author
79 
80If the handler isn't available, ask the agent author for the framing spec before connecting.
81 
82## Examples
83 
84**Connect from a Python client (no browser proxy):**
85 
86```python
87import os, uuid, websockets  # requires websockets >= 12 for the additional_headers kwarg below
88 
89token = os.popen("az account get-access-token --resource https://ai.azure.com --query accessToken -o tsv").read().strip()
90url = (
91    "wss://{account}.services.ai.azure.com/api/projects/agents/endpoint/protocols/invocations_ws"
92    "?project_name={project}&agent_name={name}"
93    f"&agent_session_id={uuid.uuid4().hex}"
94    "&foundry_features=HostedAgents=V1Preview"
95)
96 
97# websockets >= 12 uses `additional_headers`; older versions (<12) expect `extra_headers`.
98async with websockets.connect(url, additional_headers={"Authorization": f"Bearer {token}"}) as ws:
99    await ws.send(b"<first frame in your wire format>")
100    async for frame in ws:
101        ...  # frame is bytes (binary) or str (text) depending on what the container sends
102```
103 
104**Connect from a browser** — terminate a local WebSocket in a server-side proxy that injects the token, then forward frames pass-through to the upstream `wss://`.
105 
106## Error Handling
107 
108| Error | Cause | Resolution |
109|-------|-------|------------|
110| 401 / 403 on upgrade | Missing or expired Entra token | Re-mint with `az account get-access-token --resource https://ai.azure.com` |
111| 404 on upgrade | Wrong `project_name` or `agent_name`, missing preview flag, or unsupported region | Verify with `agent_get`; ensure `foundry_features=HostedAgents=V1Preview` is set; confirm the deployed version uses `protocol: invocations_ws` and that the region is supported per [Hosted Agents region availability](https://learn.microsoft.com/azure/foundry/agents/concepts/hosted-agents#region-availability) |
112| WS closes after accept | Container raised in the handler | Tail logs with `azd ai agent monitor --session-id <agent_session_id> --follow` |
113| Frames silently dropped | Wire-format mismatch (binary vs text, wrong schema) | Confirm both ends agree on framing — the platform performs no transcoding |
114| State lost on reconnect | Different `agent_session_id` used | Reuse the same `agent_session_id` to land on the same logical state inside the container |
115| Browser fails with `1006 abnormal closure` | Browser tried to connect directly with no `Authorization` | Route through a server-side proxy that adds the header |
116