Errors & rate limits
How the Inference API reports errors, the OpenAI-style error body, the status codes you'll see, and how to retry.
The Inference API uses conventional HTTP status codes and an OpenAI-style error body, so existing OpenAI clients surface errors the way you expect.
Error body
Every error response has a JSON body with an error object carrying a
human-readable message, a type, and a machine-readable code, plus a
param on typed model errors:
{
"error": {
"message": "insufficient balance; add funds to continue",
"type": "insufficient_quota",
"code": "insufficient_balance"
}
}Branch on the HTTP status first; treat type and code as stable hints and
message as text for logs and humans.
Typed model errors
Some models OpenRelay hosts publish typed error codes for their own limits;
today that is openrelay/laya on Decisions.
When one of them rejects a request, you get a specific code for the limit it
hit instead of a generic rejection, and a param naming the field:
{
"error": {
"message": "too many questions in one request",
"type": "invalid_request_error",
"code": "too_many_questions",
"param": "questions"
}
}codecomes from a fixed, documented set, andmessageis OpenRelay's canonical text for that code. Nothing the model server writes reaches you.paramis a dotted path into your own request body ([n]for array indices), for examplequestions.urgency.type. It appears only when that path exists in the request you sent; otherwise the field is omitted.- Every code is a
400 invalid_request_error: fix the request, do not retry it unchanged.
The codes are family-specific, and each family's page lists the ones it can
return. A rejection that does not map to a documented code, from any model,
keeps the generic
invalid_request_error code (status 400 or 422) with the message "the
request was rejected by the model".
Status codes
| Status | type | Meaning & what to do |
|---|---|---|
400 | invalid_request_error | Malformed JSON, a missing/empty model, or a required field like max_tokens on Messages. Fix the request. |
400 | invalid_request_error (unsupported_api) | The model doesn't serve the API family you called. Check supported_apis on GET /v1/models. |
400 | invalid_request_error (family codes) | The model rejected the request for a documented limit, such as too_many_questions on Decisions. param names the field. See Typed model errors. |
400 | invalid_request_error (no_valid_records, input_too_large, input_unreadable, input_file_changed) | A batch create created nothing: no record in the input can run (no_valid_records, with rejected_records, rejected_records_file_id, rejected_count and request_counts in error), the input holds more than 50,000 records, or the input file cannot be read or changed after its upload (upload it again). |
400 | invalid_request_error (invalid_file) | A batch file upload is not JSONL; the message says why. |
401 | authentication_error | Missing, invalid, revoked, or expired API key. Check your key and the Authorization / x-api-key header. See Authentication. |
402 | insufficient_quota | Your balance is below the spending floor. Add funds: see Pricing & billing. |
402 | insufficient_quota (key_budget_exceeded) | The key's spend budget is used up for its period, or this request's estimated cost would pass what is left of it; a batch create is refused once it is used up. Wait for the reset, lower max_tokens, or have the budget raised: see Key budgets. |
403 | permission_error | The account is unavailable, paused because its available balance dropped below $0.50 (add funds to resume), under review, or being verified. Verification usually finishes within one business day; if it takes longer, or for any other case, resolve the hold or contact support. |
403 | permission_error (permission_denied, model_not_allowed) | With permissions enabled: a role, key restriction, network rule or policy refuses the request (permission_denied, with permission and denied_by), or the model is outside the key's, role's or organization's model list (model_not_allowed, with model and denied_by). Use a key that allows the model, or ask an admin. |
404 | not_found_error | The requested model doesn't exist or isn't available to you. Check GET /v1/models. |
409 | invalid_request_error (feature_disabled) | A batch create asked for a feature your organization does not have: a record carrying openrelay.workflow, or "priority": "expedited" where it is turned off. It is answered before any record is checked, and nothing was created. |
409 | invalid_request_error (idempotency_key_in_use) | An earlier batch create with the same Idempotency-Key is still running, and may create the batch. Repeat yours after Retry-After. |
409 | invalid_request_error (idempotency_key_used) | An earlier batch create with this Idempotency-Key succeeded, but its answer could not be kept. Find the batch it created with GET /v1/batches instead of repeating it. |
413 | invalid_request_error | The request body exceeds the 10 MiB limit. Send a smaller request. |
422 | invalid_request_error | The model rejected the request (e.g. parameters it doesn't accept). Fix the request. |
422 | invalid_request_error (idempotency_key_reused) | A batch create's Idempotency-Key was used with a different body. Nothing was created: use a new key for a new batch. |
429 | rate_limit_error | Too many requests. Back off and retry. |
502 | api_error | The model is temporarily unavailable. Retry with backoff. With code upstream_header_timeout, a non-streaming request ran past the limit for the model to start its response, and retrying the same request usually hits it again. For chat completions and messages the limit is 30 minutes: stream the request or ask for fewer output tokens, and raise your client's own timeout for a long non-streaming request, since many client libraries stop waiting sooner. For embeddings and decisions it is 300 seconds. |
503 | api_error | All capacity for the model is temporarily unavailable. Retry with backoff. |
503 | api_error (catalog_unavailable, flags_unavailable, orchestrator_unavailable) | A batch create could not read the model catalog or your organization's feature settings, or could not start the batch. Retry with backoff. |
Error codes you may see
code values, by status: missing_api_key, invalid_api_key,
revoked_api_key, expired_api_key (401); insufficient_balance,
key_budget_exceeded (402);
account_suspended, account_pending_verification, account_unavailable,
permission_denied, model_not_allowed (403); model_not_found (404);
unsupported_api (400, the model doesn't serve the API family you called);
the typed model error codes (400);
invalid_idempotency_key (400), idempotency_key_in_use and
idempotency_key_used (409) and idempotency_key_reused (422), on a batch
create sent with an
Idempotency-Key; on a batch create, no_valid_records, input_too_large,
input_unreadable, input_file_changed (400), feature_disabled (409),
catalog_unavailable, flags_unavailable, orchestrator_unavailable (503);
on a batch file upload, invalid_file (400); payload_too_large (413);
rate_limit_error (429); upstream_error, upstream_header_timeout (502);
upstream_unavailable (503).
Rate limiting
A 429 reflects temporary rate limiting, not a per-account quota. OpenRelay
does not currently impose per-account request-rate (RPM/TPM) limits. Treat
it as transient: back off and retry.
Retrying
Retry safely
Retry only on 429, 502, and 503: these are transient. Use exponential
backoff with jitter. Do not retry 400, 401, 402, 403, 404, or
413 unchanged: fix the request, the key, the balance, the key's budget, or
the model first.
# Retry ONLY on transient failures (429, 502, 503) with exponential backoff.
# Any other status, success or a client error like 401/402, stops the loop.
for delay in 1 2 4 8 0; do
code=$(curl -sS -o /tmp/resp.json -w '%{http_code}' \
https://inference.openrelay.inc/v1/chat/completions \
-H "Authorization: Bearer $OPENRELAY_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"openrelay/gpt-oss-120b","messages":[{"role":"user","content":"hi"}]}')
if [ "$code" = "429" ] || [ "$code" -ge 500 ]; then
sleep "$delay"; continue # transient: back off and retry
fi
break # 2xx success, or a non-retryable client error
done
cat /tmp/resp.jsonStreaming errors
For a streaming request, the HTTP status is sent
before any tokens. If the stream is interrupted mid-response, the gateway emits a
final SSE error frame followed by data: [DONE], so a cut-off stream is never
mistaken for a complete one. Interrupted streams are not billed.