Skip to content
OpenRelay is in early access, and the /v1 API is stable. New capabilities ship in the changelog.
Skip to Content
GuidesInference APIsAsync API

Jobs

The async job object, its statuses and counts, how to create, list, get and cancel one, and how to retry a create safely.

Async is in beta

Shapes and limits can still change before general availability.

A job is one submission: a window, a set of request lines and the state of working through them. A single request is a job of one.

Create

POST /v1/async/jobs takes:

FieldRequiredMeaning
completion_windowyes1h or 24h. It sets the deadline and the price.
requestsone of the twoUp to 1,000 inline request lines.
input_file_idone of the twoAn uploaded JSONL file of request lines (purpose batch).
metadatanoUp to 32 string pairs, echoed back on the job.

Each line is {"custom_id", "method", "url", "body"}. custom_id is yours and unique in the job. method is POST and may be left out. url is /v1/chat/completions, /v1/messages or /v1/embeddings, and body is what that endpoint takes, including the model. Streaming is not supported.

Every line is checked when the job is created, with the Batch API's record rules. A line that breaks one is rejected and the rest are accepted. The answer carries rejected_count, the first 100 rejections in rejected_records and, when any line was rejected, rejected_records_file_id naming a file with all of them. A rejected line never runs, is never billed and counts as failed. If no line can run, the answer is 400 no_valid_records and no job is created.

Retrying a create

Send an Idempotency-Key header (1 to 512 bytes, such as a UUID) and resend it on every retry of the same create. A repeat with the same key and body gets the first answer back with Idempotent-Replayed: true and creates nothing. The same key with a different body answers 422, and a repeat while the first is still running answers 409; repeat it a second later.

The job object

{
  "id": "ajob_01JB7ZK3M2X8Q4V",
  "object": "async.job",
  "completion_window": "24h",
  "status": "running",
  "created_at": 1793793600,
  "started_at": 1793793606,
  "completed_at": null,
  "deadline_at": 1793880000,
  "request_counts": {"total": 5000, "completed": 3690, "failed": 36, "expired": 0, "cancelled": 0},
  "usage": {"input_tokens": 6312004, "cached_input_tokens": 120440, "output_tokens": 811302},
  "cost_nano_usd": 879215000,
  "input_file_id": "file-a1b2c3",
  "output_file_id": null,
  "error_file_id": null,
  "metadata": {"run": "nightly-labels"}
}
  • deadline_at is created_at plus the window. A request not started by then expires.
  • cost_nano_usd is what has been billed so far, in billionths of a dollar. It settles as each group of requests finishes, so it climbs during the run.
  • output_file_id and error_file_id are always null today: a finished job leaves no output file. Read its results from the results endpoint.
  • request_counts counts requests by outcome; total minus the outcomes is what has not finished.

Statuses

StatusMeaning
queuedCreated, not started.
runningAt least one request has started.
cancellingA cancel is waiting for in-flight requests to finish.
completedEvery request reached an outcome, whatever it was. A job past its deadline ends completed, with its unstarted requests counted expired.
cancelledCancelled.
failedThe input could not be read at all. A bad line does not fail the job.

List and get

GET /v1/async/jobs/{id} returns one job. GET /v1/async/jobs returns your organization's jobs, newest first, limit up to 100, with after set to the previous page's last_id.

Add status=completed (or any one status) to list only those jobs. A filtered request reads a bounded number of jobs, so a page can hold fewer than limit, or none, with has_more true. Keep paging with after set to last_id, which is then the last job the request read and may not be on the page.

Cancel

POST /v1/async/jobs/{id}/cancel stops starting the job's requests. Requests already running finish and are charged. The rest end cancelled at no charge. A job that has not started ends cancelled at once. A running one answers cancelling until its in-flight requests finish. Cancelling a job that has already ended answers 409 async_job_not_cancellable.

Retention

A job and its files are kept for 30 days from creation, then deleted automatically.

On this page