Overview
The OpenRelay Inference API, an OpenAI-compatible chat completions API and, for supporting models, an Anthropic-compatible Messages API.
The OpenRelay Inference API is a hosted inference gateway that speaks two API families: an OpenAI-compatible Chat Completions API, and, for models that support it, an Anthropic-compatible Messages API. Point your existing OpenAI or Anthropic client at it, change the base URL and the API key, and your code works unchanged.
Base URL
All inference requests, including the Batch API, go to:
https://inference.openrelay.incThis is a separate host from api.openrelay.inc, the control plane that
manages VMs, clusters, organizations, billing, and the
model catalog. Both accept the same vl_ API key;
only the routes differ. If an inference call to api.openrelay.inc returns
404, you are on the wrong host.
Endpoints are versioned under /v1, mirroring the OpenAI API:
curl https://inference.openrelay.inc/v1/chat/completions \
-H "Authorization: Bearer vl_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"model": "openrelay/gpt-oss-120b",
"messages": [{ "role": "user", "content": "Hello!" }]
}'What you get
- OpenAI-compatible — the
POST /v1/chat/completionsrequest and response shapes match the OpenAI Chat Completions API, so existing SDKs and tools work as-is. - Anthropic-compatible, for models that support it:
POST /v1/messagesmatches the Anthropic Messages API, so the Anthropic SDKs and Claude Code work as-is too. See Messages. - Streaming — set
"stream": truefor token-by-token Server-Sent Events, including a final chunk (Chat Completions) ormessage_delta/message_stopevents (Messages) that reportusage. See Streaming. - Prepaid, per-token billing — usage is metered per token with distinct
input, cached-input, cache-write, and output rates, and reported in every
response's
usageobject. See Pricing & billing.
OpenRelay model ids
Requests and responses use OpenRelay's public model ids (for example
openrelay/gpt-oss-120b). The model field in every response is always the
public id you requested. See Models.
Endpoints
| Endpoint | Auth | Purpose |
|---|---|---|
POST /v1/chat/completions | API key | Generate a chat completion (unary or streaming). |
POST /v1/messages | API key | Generate a message, Anthropic-shape (unary or streaming), for models that support it. |
GET /v1/models | API key | List the models available to you. |
POST /v1/batches | API key | Run a large job of requests asynchronously at a reduced rate. See Batch API. |
GET /healthz | none | Service health check. |
Not every model supports both families. Check a model's supported_apis
field from GET /v1/models. See Models.
Get started
Quickstart
Your first request with curl — unary and streaming.
Authentication
Use a vl_ API key with the headers the gateway accepts.
Chat Completions
The POST /v1/chat/completions request and response shapes.
Messages
The POST /v1/messages request and response shapes, Anthropic-compatible.
Using OpenAI SDKs
Drop-in setup for the Python and Node OpenAI libraries.
Batch API
Run large jobs of requests asynchronously within 24 hours at a reduced rate.