Skip to content
OpenRelay is in early access, and the /v1 API is stable. New capabilities ship in the changelog.
PricingDashboard
GuidesAPI ReferenceCLI
API ReferenceInferenceChat Completions

Generate a chat completion

Generates a model response for a conversation in the OpenAI Chat Completions shape, from any model whose supported APIs include chat completions. The hand-written `orl chat` is the friendlier front for this operation: model name completion, a readable transcript, and streaming output rendered to the terminal. Requires a funded balance. Set "stream" to true to receive the completion incrementally as Server-Sent Events, ending with a `data: [DONE]` sentinel. The request body is capped at 10 MiB.

Required permission

Requires the inference.invoke permission. See Permissions for the roles and API key permissions that include it.

Command line

The orl CLI wraps this endpoint:

orl chat-completions create --messages <messages> --model <model>
POST
/v1/chat/completions

Authorization

apiKey
headerAuthorizationBearer <token>

OpenRelay API key. Send it as Authorization: Bearer or_… (legacy vl_… keys still work).

Request Body

application/json
  1. body

An OpenAI-compatible chat completions request. Opaque: unlisted OpenAI sampling and control parameters (temperature, top_p, stop, presence_penalty, frequency_penalty, seed, stream_options, ...) are forwarded as sent; unsupported ones are ignored rather than rejected.

model*string

An OpenRelay public model id, e.g. openrelay/gpt-oss-120b.

messages*array<>

The conversation so far: {role, content} objects (system, user, assistant).

max_completion_tokens?integer

Upper bound on the number of tokens to generate.

stream?boolean

Stream the response as Server-Sent Events.

[key: string]?any

Response Body

OK

  1. response

An OpenAI chat.completion object (opaque; additionalProperties carries the rest of the dialect).

id?string
object?"chat.completion"
Value in"chat.completion"
model?string

Always the public OpenRelay model id requested.

choices?array<>
usage?
openrelay?

What the request is billed, on every online inference response.

[key: string]?any
Example Request
curl -X POST 'https://inference.openrelay.inc/v1/chat/completions' \  -H 'Content-Type: application/json' \  -d '{  "model": "string",  "messages": [    {}  ]}'
Example Response
{  "id": "string",  "object": "chat.completion",  "model": "string",  "choices": [    {}  ],  "usage": {},  "openrelay": {    "cost_nano_usd": 0,    "priced_at": "string"  }}

Overview

Generate a model response for a conversation, OpenAI-compatible: the core online inference family, with `orl chat` as its friendly front. Served by the inference endpoint (inference.openrelay.inc).

Overview

Generate a model response in the Anthropic Messages shape for models that support it: a drop-in for the Anthropic SDKs and Claude Code. Served by the inference endpoint (inference.openrelay.inc).