Streaming

Set "stream": true on a chat completion to receive the response token-by-token as Server-Sent Events (SSE) instead of waiting for the whole completion.

The SSE formatCopy Anchor Link

The response is sent with Content-Type: text/event-stream. Each event is a single data: line whose payload is a JSON chat.completion.chunk object, separated from the next by a blank line:

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"openrelay/gpt-oss-120b","choices":[{"index":0,"delta":{"role":"assistant"}}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"openrelay/gpt-oss-120b","choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"openrelay/gpt-oss-120b","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":"stop"}]}

Concatenate the choices[0].delta.content fragments across chunks to assemble the full message. As with unary responses, the model field on every chunk is the public OpenRelay id you requested.

Usage and cost in the final chunkCopy Anchor Link

The stream always ends with a final chunk that carries the request's usage totals (and typically an empty choices array), and what the request costs in an openrelay object, so you can account for tokens and spend without a second request:

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","model":"openrelay/gpt-oss-120b","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":9,"total_tokens":21},"openrelay":{"cost_nano_usd":7200,"priced_at":"online"}}

cost_nano_usd is what the request is billed, in nanoUSD (see Cost on every response). It holds for a stream that runs to completion: one cut off or ending with an error is not billed. A stream sends no OpenRelay-Cost-Nano-USD header, since its headers go out before any usage exists.

With "stream_options": {"continuous_usage_stats": true}, a model that supports it reports the usage so far on every chunk. Each of those chunks then carries the cost of the usage so far, and the last one is the bill.

The [DONE] sentinelCopy Anchor Link

After the final chunk, the stream emits the OpenAI-style termination sentinel and closes:

data: [DONE]

Stop reading when you receive [DONE].

Consuming the streamCopy Anchor Link

curl -N https://inference.openrelay.inc/v1/chat/completions \
  -H "Authorization: Bearer $OPENRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openrelay/gpt-oss-120b",
    "stream": true,
    "messages": [{ "role": "user", "content": "Tell me a joke." }]
  }'

The OpenAI SDKs parse the SSE stream and the [DONE] sentinel for you. See Using OpenAI SDKs for full setup.