Overview
The OpenRelay Inference API, with OpenAI-compatible chat completions and embeddings, an Anthropic-compatible Messages API, and typed decisions.
The OpenRelay Inference API is the entry point to OpenRelay's inference delivery network, a hosted inference gateway that speaks four API families: an OpenAI-compatible Chat Completions API, an Anthropic-compatible Messages API for models that support it, an OpenAI-compatible Embeddings API, and a Decisions API for typed classification. Point your existing OpenAI or Anthropic client at it, change the base URL and the API key, and your code works unchanged.
Base URL
All inference requests, including the Batch API, go to:
https://inference.openrelay.incThis is a separate host from api.openrelay.inc, the control plane that
manages VMs, pods, organizations, billing, and the
model catalog. Both accept the same or_ API key;
only the routes differ. If an inference call to api.openrelay.inc returns
404, you are on the wrong host.
Endpoints are versioned under /v1, mirroring the OpenAI API:
curl https://inference.openrelay.inc/v1/chat/completions \
-H "Authorization: Bearer or_your_api_key" \
-H "Content-Type: application/json" \
-d '{
"model": "openrelay/gpt-oss-120b",
"messages": [{ "role": "user", "content": "Hello!" }]
}'What you get
- OpenAI-compatible: the
POST /v1/chat/completionsrequest and response shapes match the OpenAI Chat Completions API, so existing SDKs and tools work as-is. - Anthropic-compatible, for models that support it:
POST /v1/messagesmatches the Anthropic Messages API, so the Anthropic SDKs and Claude Code work as-is too. See Messages. - Embeddings:
POST /v1/embeddingsmatches the OpenAI Embeddings API. See Embeddings. - Decisions:
POST /v1/decisionsanswers typed questions (a choice, a score, or yes/no) about an input with probabilities and a confidence. See Decisions. - Streaming: set
"stream": truefor token-by-token Server-Sent Events, including a final chunk (Chat Completions) ormessage_delta/message_stopevents (Messages) that reportusage. See Streaming. - Prepaid, per-token billing: usage is metered per token with distinct
input, cached-input, cache-write, and output rates (embeddings and decisions
bill input only), and reported in every response's
usageobject. See Pricing & billing.
OpenRelay model ids
Requests and responses use OpenRelay's public model ids (for example
openrelay/gpt-oss-120b). The model field in every response is always the
public id you requested. See Models.
Three ways to call a model
Every model is priced per token, and the rate depends on how fast you need the answer:
| Tier | How it runs | Price vs Sync | How to call it |
|---|---|---|---|
| Sync | Real time. The response (or stream) comes back on the request. | Standard rate | POST /v1/chat/completions, /v1/messages, /v1/embeddings, /v1/decisions |
| Async | Queued. Results are delivered later, not on the request. | 20% off | Coming soon, early access only. Email sales@openrelay.inc. |
| Batch | A JSONL file of requests, completed within 24 hours. | 50% off at standard priority | POST /v1/batches. See Batch API. |
The discount applies to input, cached input, and output tokens alike. See Pricing & billing for the exact rules and a worked example.
Endpoints
| Endpoint | Auth | Purpose |
|---|---|---|
POST /v1/chat/completions | API key | Generate a chat completion (unary or streaming). |
POST /v1/messages | API key | Generate a message, Anthropic-shape (unary or streaming), for models that support it. |
POST /v1/embeddings | API key | Embed text as vectors, OpenAI-shape (unary). See Embeddings. |
POST /v1/decisions | API key | Answer typed questions about an input (unary). See Decisions. |
GET /v1/models | API key | List the models available to you. |
POST /v1/batches | API key | Run a large job of requests asynchronously, at 50% off the Sync rate. See Batch API. |
GET /healthz | none | Service health check. |
Each model serves only the families it lists. Check a model's
supported_apis field from GET /v1/models, and its endpoints map for
the path to call. See Models.
Get started
Quickstart
Your first request with curl: unary and streaming.
Authentication
Use an or_ API key with the headers the gateway accepts.
Chat Completions
The POST /v1/chat/completions request and response shapes.
Messages
The POST /v1/messages request and response shapes, Anthropic-compatible.
Embeddings
The POST /v1/embeddings request and response shapes, OpenAI-compatible.
Decisions
Typed choice, score, and yes/no answers with POST /v1/decisions.
Using OpenAI SDKs
Drop-in setup for the Python and Node OpenAI libraries.
Batch API
Run large jobs of requests within 24 hours at 50% off the Sync rate.
Managed Kubernetes
Kubernetes clusters whose control plane we run, with whole GPU or CPU hosts as nodes. Create a cluster and a node pool, write a kubeconfig, and use kubectl.
Quickstart
From an API key to your first chat completion, unary and streaming, using nothing but curl against the OpenAI-compatible OpenRelay API.