Skip to content
OpenRelay is in early access, and the /v1 API is stable. New capabilities ship in the changelog.
Skip to Content
GuidesInference APIsAsync API

Quickstart

Create an async job with curl, the orl CLI or the OpenAI Python SDK, watch it finish, and read the results of the requests it ran.

Async is in beta

Shapes and limits can still change before general availability.

This creates a job of two requests with a 1-hour window, then reads its results. Every request goes to inference.openrelay.inc. Set your API key first:

export OPENRELAY_API_KEY="or_your_api_key"

Use a model that is offered for Async in the window you pick; a line that names another is rejected with model_not_offered, and the message lists the models offered. GET /v1/models lists each model's windows in async_windows.

Create the job

curl -s https://inference.openrelay.inc/v1/async/jobs \
  -H "Authorization: Bearer $OPENRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: quickstart-1" \
  -d '{
    "completion_window": "1h",
    "metadata": {"run": "quickstart"},
    "requests": [
      {"custom_id": "req-1", "url": "/v1/chat/completions",
       "body": {"model": "openrelay/glm-5.3-flash",
                "messages": [{"role": "user", "content": "Name three primary colors."}]}},
      {"custom_id": "req-2", "url": "/v1/chat/completions",
       "body": {"model": "openrelay/glm-5.3-flash",
                "messages": [{"role": "user", "content": "Name three prime numbers."}]}}
    ]
  }'

The answer is the job, queued and with a deadline_at one hour out:

{
  "id": "ajob_01JB7ZK3M2X8Q4V",
  "object": "async.job",
  "completion_window": "1h",
  "status": "queued",
  "request_counts": {"total": 2, "completed": 0, "failed": 0, "expired": 0, "cancelled": 0},
  "deadline_at": 1793797200,
  "metadata": {"run": "quickstart"}
}

Watch it

curl -s https://inference.openrelay.inc/v1/async/jobs/ajob_01JB7ZK3M2X8Q4V \
  -H "Authorization: Bearer $OPENRELAY_API_KEY"
# or: orl async job get ajob_01JB7ZK3M2X8Q4V

status goes queued, running, then completed. request_counts and cost_nano_usd climb while it runs. Rather than polling, you can subscribe a webhook to async.job.completed.

Read the results

curl -s "https://inference.openrelay.inc/v1/async/jobs/ajob_01JB7ZK3M2X8Q4V/results?status=completed" \
  -H "Authorization: Bearer $OPENRELAY_API_KEY"
# or: orl async job-result list ajob_01JB7ZK3M2X8Q4V --status completed

Each result carries your custom_id, its status, the response the Sync endpoint would have returned and what the request cost. See Results for paging.

Many requests

Past 1,000 requests, upload a JSONL file of request lines with the Files API (purpose batch) and create the job with input_file_id instead of requests. A file the Batch API accepts works unchanged, as long as every line has a url from the list in the overview.

orl async job create --completion-window 24h --input-file-id file-a1b2c3

On this page