OpenRelay API Documentation

How the Inference API reports errors — the OpenAI-style error body, the status codes you'll see, and how to retry.

The Inference API uses conventional HTTP status codes and an OpenAI-style error body, so existing OpenAI clients surface errors the way you expect.

Error body

Every error response has a JSON body with an error object carrying a human-readable message, a type, and a machine-readable code:

{
  "error": {
    "message": "insufficient balance; add funds to continue",
    "type": "insufficient_quota",
    "code": "insufficient_balance"
  }
}

Branch on the HTTP status first; treat type and code as stable hints and message as text for logs and humans.

Status	`type`	Meaning & what to do
`400`	`invalid_request_error`	Malformed JSON, a missing/empty `model`, or a required field like `max_tokens` on Messages. Fix the request.
`400`	`invalid_request_error` (`unsupported_api`)	The model doesn't serve the API family you called. Check `supported_apis` on `GET /v1/models`.
`401`	`authentication_error`	Missing, invalid, revoked, or expired API key. Check your key and the `Authorization` / `x-api-key` header. See Authentication.
`402`	`insufficient_quota`	Your balance is below the spending floor. Add funds — see Pricing & billing.
`403`	`permission_error`	The account is unavailable, suspended for balance, or under review. Resolve the hold or contact support.
`404`	`not_found_error`	The requested `model` doesn't exist or isn't available to you. Check `GET /v1/models`.
`413`	`invalid_request_error`	The request body exceeds the 10 MiB limit. Send a smaller request.
`422`	`invalid_request_error`	The model rejected the request (e.g. parameters it doesn't accept). Fix the request.
`429`	`rate_limit_error`	Too many requests. Back off and retry.
`502`	`api_error`	The model is temporarily unavailable. Retry with backoff.
`503`	`api_error`	All capacity for the model is temporarily unavailable. Retry with backoff.

code values, by status: missing_api_key, invalid_api_key, revoked_api_key, expired_api_key (401); insufficient_balance (402); account_suspended, account_unavailable (403); model_not_found (404); unsupported_api (400, the model doesn't serve the API family you called); payload_too_large (413); rate_limit_error (429); upstream_error (502); upstream_unavailable (503).

Rate limiting

A 429 reflects temporary rate limiting, not a per-account quota. OpenRelay does not currently impose per-account request-rate (RPM/TPM) limits. Treat it as transient: back off and retry.

Retrying

Retry safely

Retry only on 429, 502, and 503 — these are transient. Use exponential backoff with jitter. Do not retry 400, 401, 402, 403, 404, or 413 unchanged: fix the request, the key, the balance, or the model first.

# Retry ONLY on transient failures (429, 502, 503) with exponential backoff.
# Any other status — success or a client error like 401/402 — stops the loop.
for delay in 1 2 4 8 0; do
  code=$(curl -sS -o /tmp/resp.json -w '%{http_code}' \
      https://inference.openrelay.inc/v1/chat/completions \
      -H "Authorization: Bearer $OPENRELAY_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"model":"openrelay/gpt-oss-120b","messages":[{"role":"user","content":"hi"}]}')
  if [ "$code" = "429" ] || [ "$code" -ge 500 ]; then
    sleep "$delay"; continue   # transient — back off and retry
  fi
  break                        # 2xx success, or a non-retryable client error
done
cat /tmp/resp.json

Streaming errors

For a streaming request, the HTTP status is sent before any tokens. If the stream is interrupted mid-response, the gateway emits a final SSE error frame followed by data: [DONE], so a cut-off stream is never mistaken for a complete one. Interrupted streams are not billed.

Errors & rate limits

Error body

Status codes

Retrying

Streaming errors

On this page