Skip to content
OpenRelay is in early access, and the /v1 API is stable. New capabilities ship in the changelog.
Skip to Content
GuidesInference APIsAsync API

Results

The shape of an async result, how to page through the results of a job while it runs and after it ends, and why there is no output file.

Async is in beta

Shapes and limits can still change before general availability.

A result

One result per request, in the same shape on a result page and inline in a webhook:

{
  "custom_id": "ticket-T-123",
  "status": "completed",
  "late": false,
  "started_at": 1793793611,
  "completed_at": 1793793639,
  "response": {"status_code": 200, "body": {"object": "chat.completion", "choices": [], "usage": {}}},
  "error": null,
  "cost_nano_usd": 397280
}
FieldMeaning
statuscompleted, failed, expired or cancelled.
latetrue for a completed request that started before the deadline and finished after it. It is charged normally.
responseA completed request's answer: the status_code and the body the Sync endpoint returns for the same request, as compact JSON. null otherwise.
errorWhy a request did not complete, with a code (see Limits and errors) and a message. null when it did.
cost_nano_usdWhat the request was charged, in billionths of a dollar. 0 for a request that did not complete. null for a completed request whose price is not known; a job's results add up to its cost only when none is null.
lineOnly on a request the create rejected: its line in the input, from 1.

expired and cancelled mean the request never started. A request that was sent and may have run but could not finish ends failed with request_failed, not expired.

Paging while the job runs

GET /v1/async/jobs/{id}/results returns finished results, a page at a time, and works while the job is running.

curl -s "https://inference.openrelay.inc/v1/async/jobs/$JOB/results?limit=100" \
  -H "Authorization: Bearer $OPENRELAY_API_KEY"
# or: orl async job-result list $JOB --limit 100
{"object": "list", "data": [], "next_cursor": "…", "has_more": true}
  • after takes the previous page's next_cursor. A cursor pages only the job, and the status filter, it was read under.
  • limit is up to 1,000, default 100.
  • status=completed returns only completed results. status=failed returns every result that did not complete: failed, expired or cancelled.
  • has_more is true while the job is live and after any full page, and false once the job has ended and a page comes back short. On a full page, send the response's next_cursor as the next after: the same cursor returns that same page again. Reuse a cursor only after a page that was not full (a live job's short page, to wait for more results). Stop when has_more is false.

Results arrive a group of requests at a time, in group order, as each group finishes. A page ends at the first group still running, and the same cursor returns that group's results once it finishes, even if a later group finished first. Within a group, completed results come before the rest. So a running job's request_counts can run ahead of what its pages show, and a page can hold fewer than limit results, or none, with has_more true. A page reads at most 32 MB of results plus the one result that crosses that budget; a result is never split.

A job cancelled before its requests were grouped counts them cancelled but has no result for each; its pages show only the requests the create rejected.

No output file yet

A finished job does not leave an output file: output_file_id and error_file_id stay null. Read every result through the results endpoint (GET /v1/async/jobs/{id}/results, or orl async job-result list) with status=completed for the completed results and status=failed for the rest, or take them inline from the webhook event of a small job. A request the create rejected is in the file rejected_records_file_id names, and also on the results pages.

orl async job-result list reads up to 96 MiB in one page. Past that, or in an MCP tool, which reads 8 MiB, the read fails: ask for a smaller page with limit and read on with after.

On this page