Create an async job
Submit requests as a job, for delivery within 1 hour (20% off the Sync price) or within 24 hours (50% off Sync). Send them inline (requests, up to 1,000) or as an uploaded input file (input_file_id); provide exactly one of the two. Each line names its url (/v1/chat/completions, /v1/messages or /v1/embeddings) and the body that endpoint takes, and a job may mix models and endpoints. Every line is checked when the job is created, with the Batch API's record rules: a line that breaks one is rejected and the rest are accepted. The answer counts the rejections in rejected_count, lists the first 100 in rejected_records, and rejected_records_file_id names a file holding all of them; a rejected line never runs, is never billed, and counts as failed. When no line can run, the answer is 400 no_valid_records and nothing is created. A request not started before deadline_at expires at no charge; one that started finishes and is charged. A 1-hour job holds up to 10,000 requests and a 24-hour job up to 50,000.
Required permission
Requires the async.write permission. See Permissions for the roles and API key permissions that include it.
Command line
The orl CLI wraps this endpoint:
orl async job create --completion-window <completion-window>apiKeyAuthorizationBearer <token>OpenRelay API key. Send it as Authorization: Bearer or_… (legacy vl_… keys still work).
Idempotency-Key?stringMakes a retried create safe. Choose one value per create you mean to make, 1 to 512 bytes (bytes, not characters), such as a UUID, and send it again on every retry of that create. The first request with a key runs, and a 2xx answer is kept for 24 hours: a repeat with the same key and the same body gets that answer back, with the same status and the header Idempotent-Replayed: true, and creates nothing. A batch create compares the body byte for byte; a VM or pod create compares the fields it decodes, so whitespace and key order do not count. A repeat while the first request is still running answers 409; repeat it a second later. The same key with a different body answers 422. An answer other than a 2xx is not kept, so a retry after a 4xx or a 5xx runs again. A 2xx too large to keep holds the key without it: every repeat answers 409 and creates nothing. Keys are scoped to the organization and the operation.
1 <= lengthapplication/json- body
completion_window*stringThe delivery window, which sets the price tier.
"1h""24h"input_file_id?stringId of an uploaded JSONL input file (purpose batch).
requests?array<>Inline request lines, an alternative to input_file_id.
items <= 1000metadata?Key-value pairs echoed back on the job, at most 32 pairs.
The job was created. rejected_count is how many lines the create rejected; when it rejected any, the answer also carries rejected_records and rejected_records_file_id.
application/json- response
An async job.
id*stringThe job id (ajob_...).
object*"async.job""async.job"completion_window*string"1h""24h"status*string"queued""running""cancelling""completed""cancelled""failed"created_at*integerUnix seconds.
int64started_at*integerWhen the first request began running; null until then.
int64completed_at*integerWhen the job ended; null while it is live.
int64deadline_at*integercreated_at plus the window. A request not started by then expires.
int64request_counts*A job's requests by outcome; total minus the outcomes have not finished.
usage*cost_nano_usd*integerBilled so far, in nanodollars (1e-9 USD).
int64input_file_id*stringThe job's input: the uploaded file, or the file its inline requests were stored as.
output_file_id*stringAlways null today: a finished job leaves no output file. Read the results from GET /v1/async/jobs/{id}/results.
error_file_id*stringAlways null today: a finished job leaves no error file. Read the failed results from GET /v1/async/jobs/{id}/results with status=failed.
metadata*rejected_count?integerOnly on the create answer, how many lines the create rejected.
int64rejected_records?array<>Only on the create answer, and only when the create rejected lines: the first 100 rejected lines, in input order.
items <= 100rejected_records_file_id?stringOnly on the create answer, and only when the create rejected lines: the file holding every rejected line.
curl -X POST 'https://inference.openrelay.inc/v1/async/jobs' \ -H 'Content-Type: application/json' \ -d '{ "completion_window": "1h"}'{ "id": "string", "object": "async.job", "completion_window": "1h", "status": "queued", "created_at": 0, "started_at": 0, "completed_at": 0, "deadline_at": 0, "request_counts": { "total": 0, "completed": 0, "failed": 0, "expired": 0, "cancelled": 0 }, "usage": { "input_tokens": 0, "cached_input_tokens": 0, "output_tokens": 0 }, "cost_nano_usd": 0, "input_file_id": "string", "output_file_id": "string", "error_file_id": "string", "metadata": { "property1": "string", "property2": "string" }, "rejected_count": 0, "rejected_records": [ { "line": 0, "custom_id": "string", "code": "invalid_json", "message": "string" } ], "rejected_records_file_id": "string"}List async jobs GET
Your organization's async jobs, newest first, a page at a time. Add status to list only the jobs with that status. A request with status reads a bounded number of jobs, so a page can hold fewer than limit, or none, with has_more true. Page on with after set to its last_id, which is then the last job the request read and may not be on the page.
Get an async job GET
The job, its status, request counts, token usage and the cost billed so far, readable while it runs and after it ends.