Quickstart
Create an async job with curl, the orl CLI or the OpenAI Python SDK, watch it finish, and read the results of the requests it ran.
Async is in beta
Shapes and limits can still change before general availability.
This creates a job of two requests with a 1-hour window, then reads its
results. Every request goes to inference.openrelay.inc. Set your API key
first:
export OPENRELAY_API_KEY="or_your_api_key"Use a model that is offered for Async in the window you pick; a line that names
another is rejected with model_not_offered, and the message lists the models
offered. GET /v1/models lists each model's windows in async_windows.
Create the job
curl -s https://inference.openrelay.inc/v1/async/jobs \
-H "Authorization: Bearer $OPENRELAY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: quickstart-1" \
-d '{
"completion_window": "1h",
"metadata": {"run": "quickstart"},
"requests": [
{"custom_id": "req-1", "url": "/v1/chat/completions",
"body": {"model": "openrelay/glm-5.3-flash",
"messages": [{"role": "user", "content": "Name three primary colors."}]}},
{"custom_id": "req-2", "url": "/v1/chat/completions",
"body": {"model": "openrelay/glm-5.3-flash",
"messages": [{"role": "user", "content": "Name three prime numbers."}]}}
]
}'The answer is the job, queued
and with a deadline_at one hour out:
{
"id": "ajob_01JB7ZK3M2X8Q4V",
"object": "async.job",
"completion_window": "1h",
"status": "queued",
"request_counts": {"total": 2, "completed": 0, "failed": 0, "expired": 0, "cancelled": 0},
"deadline_at": 1793797200,
"metadata": {"run": "quickstart"}
}Watch it
curl -s https://inference.openrelay.inc/v1/async/jobs/ajob_01JB7ZK3M2X8Q4V \
-H "Authorization: Bearer $OPENRELAY_API_KEY"
# or: orl async job get ajob_01JB7ZK3M2X8Q4Vstatus goes queued, running, then completed. request_counts and
cost_nano_usd climb while it runs. Rather than polling, you can
subscribe a webhook to async.job.completed.
Read the results
curl -s "https://inference.openrelay.inc/v1/async/jobs/ajob_01JB7ZK3M2X8Q4V/results?status=completed" \
-H "Authorization: Bearer $OPENRELAY_API_KEY"
# or: orl async job-result list ajob_01JB7ZK3M2X8Q4V --status completedEach result carries your custom_id, its status, the response the Sync
endpoint would have returned and what the request cost. See
Results for paging.
Many requests
Past 1,000 requests, upload a JSONL file of request lines with the
Files API (purpose batch) and create the job
with input_file_id instead of requests. A file the Batch API accepts works
unchanged, as long as every line has a url from the list in the
overview.
orl async job create --completion-window 24h --input-file-id file-a1b2c3