Limits and errors
The size limits of an async job, the errors a create can answer with, and the codes a failed request carries in its result.
Async is in beta
Shapes and limits can still change before general availability.
Limits
| 1 hour | 24 hours | |
|---|---|---|
| Requests per job | 10,000 | 50,000 |
| Inline requests per create | 1,000 | 1,000 |
| Input file | 200 MB | 200 MB |
| One model call | 30 minutes | 30 minutes |
| Retention of a job and its files | 30 days | 30 days |
A request answer over 64 MB fails that request with response_too_large.
Your key's budget and balance apply as for any
inference. The routes need Batch: Read (list, get, results) and Batch:
Write (create, cancel) permissions.
There is no capacity refusal: every job is accepted, and a request that does not start before its deadline expires at no charge.
Errors at create
The answer is the error shape with these codes:
| Status | Code | When |
|---|---|---|
| 400 | invalid_request_error | A bad completion_window, both or neither of input_file_id and requests, or more than 1,000 inline requests. |
| 400 | input_too_large | More lines than the window allows. |
| 400 | no_valid_records | No line can run. A line is judged on its own, and the create fails only when every line is rejected. |
| 400 | input_unreadable, input_file_changed | The input file's content is missing or is not what its upload recorded. Upload it again. |
| 402 | insufficient_balance, key_budget_exceeded | As for Sync inference. |
| 403 | The key may not use Async, or a hold applies, as for Sync inference. | |
| 404 | Async is not turned on for the organization. | |
| 409 | idempotency_key_in_use | A create with the same Idempotency-Key is still running. |
| 422 | idempotency_key_reused | The key was used with a different body. |
| 503 | catalog_unavailable, orchestrator_unavailable | Retry. |
Why a line is rejected
A rejected line is counted in rejected_count and listed with a code:
| Code | Meaning |
|---|---|
streaming_unsupported | The body asks to stream. |
invalid_url | The url is not /v1/chat/completions, /v1/messages or /v1/embeddings. |
model_not_offered | The model is not offered for Async in this window. The message names the window and lists the models offered. |
model_not_allowed | The key may not use the model. |
invalid_body | The body is not a JSON object, repeats a key the create reads (model, stream, tools, tool_choice, openrelay), or carries an openrelay extension an async job does not run. |
invalid_request | The line failed validation. |
Why a request does not complete
A result with status other than completed carries an error.code:
| Code | Status | Meaning | Charged |
|---|---|---|---|
deadline_expired | expired | It never started before deadline_at. | No |
cancelled | cancelled | The job was cancelled before it started. | No |
model_error | failed | The model answered non-2xx. error.status_code and error.body carry its answer. | No |
model_unavailable | failed | The model was withdrawn or disabled for Async while the request waited. | No |
request_failed | failed | The request was sent and may have run, then could not finish. | No |
request_time_limit | failed | The model call hit the 30 minute limit. | No |
response_too_large | failed | The answer was over the size limit. | No |
Only a completed request is charged. See Pricing.
Agents
What an async request can do for an agent today. Tool calls come back for your own loop; OpenRelay-run tool loops and workflows are not available in Async.
Pricing
Within 1 hour is 20% off Sync, within 24 hours 50% off. Only completed requests are charged; one not started by the deadline expires unbilled.