Responses (OpenAI)
POST /v1/responses, the OpenAI Responses endpoint in stateless mode, the fields it refuses, its typed streaming events, and how responses are billed.
Generate a model response using the OpenAI Responses API shape. Use it with the
OpenAI SDKs' responses.create and with Codex.
POST https://inference.openrelay.inc/v1/responsesOpenRelay serves Responses stateless: every request carries the whole
conversation, and nothing is stored between requests. store is always
false, whatever the request sets.
Which models support this
Check a model's supported_apis field from
GET /v1/models. A model serves this endpoint only
when it lists responses:
{
"id": "openrelay/<model>",
"supported_apis": ["chat_completions", "responses"]
}A request to a model that doesn't list responses returns 400 unsupported_api.
Models that serve Responses have a context window of at least 32,768 tokens.
Request
curl https://inference.openrelay.inc/v1/responses \
-H "Authorization: Bearer $OPENRELAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "openrelay/<model>",
"instructions": "Answer in one sentence.",
"input": [{ "role": "user", "content": "What is prefix caching?" }],
"max_output_tokens": 256
}'The fields of the OpenAI Responses request are passed to the model, with these exceptions.
| Field | What happens |
|---|---|
store | Always set to false. |
previous_response_id, conversation | Refused with 400 unsupported_parameter. Send the earlier turns, including the model's output items and your tool results, in input. |
background: true | Refused with 400 unsupported_parameter. Stream the response instead. |
prompt, context_management | Refused with 400 unsupported_parameter. |
item_reference and compaction_trigger input items | Refused with 400 unsupported_parameter. Send the item itself. |
input_file parts and file_id | Refused with 400 unsupported_parameter. |
include | Only reasoning.encrypted_content and message.output_text.logprobs are kept; other values are removed. |
tools of type function, custom, namespace, and tool_search with "execution": "client" | Passed through. The model emits the call and your client runs it. |
tools of type web_search | Removed from the request. The model answers without search. |
Other hosted tools (file_search, code_interpreter, mcp, image_generation, and the rest) | Refused with 400 unsupported_tool_type. |
prompt_cache_key is passed through to the model. It does not pin a request
to a replica. A request whose prompt prefix is already cached on the replica
that serves it can reuse that cache, and only the cached tokens the model
reports bill at the cached-input rate.
Response
A unary request returns a Response object. The model's text is in the
output_text parts of the message items in output, and model is the
public OpenRelay id you requested:
{
"id": "resp_…",
"object": "response",
"status": "completed",
"model": "openrelay/<model>",
"output": [
{
"type": "message",
"role": "assistant",
"content": [{ "type": "output_text", "text": "Prefix caching reuses…" }]
}
],
"usage": {
"input_tokens": 14,
"input_tokens_details": { "cached_tokens": 0 },
"output_tokens": 64,
"total_tokens": 78
}
}status is incomplete when the response stopped early, with
incomplete_details.reason saying why (for example max_output_tokens).
Streaming
Set "stream": true for typed Server-Sent Events: each event has an event:
line naming its type, such as response.output_text.delta. The stream ends
with one terminal event (response.completed, response.incomplete or
response.failed) whose response carries the final status and usage.
There is no [DONE] line. See Streaming.
Billing
Usage bills per token from usage: input_tokens at the input rate, of which
input_tokens_details.cached_tokens at the cached-input rate, and
output_tokens at the output rate.
| How the response ended | Billed |
|---|---|
completed | Yes |
incomplete (for example max_output_tokens) | Yes, for the tokens used |
failed or cancelled | No |
A stream that ends with an error event or without a terminal event | No |
The status inside the final response decides this, not the event name, so a
response.completed event whose response says failed is not billed.
Errors
Responses uses the same error envelope and status codes as the rest of the
API. See Errors. The two codes specific to this
endpoint are unsupported_parameter and unsupported_tool_type; both are
400 invalid_request_error, and message names what to change.