OpenRelay API Documentation

Run a batch end to end. Build a JSONL file, upload it, create a batch, poll to completion, and download the results, with curl, the orl CLI, or the OpenAI SDK.

This walks through a full batch three ways: raw curl, the orl CLI, and the OpenAI Python SDK (the API is OpenAI-compatible, so the SDK works unchanged with a different base URL). Every request here goes to inference.openrelay.inc, the inference host; api.openrelay.inc is the control plane and does not serve batch routes. Set your API key first:

export OPENRELAY_API_KEY="vl_your_api_key"

CLI users can run orl auth login once instead; it stores the key for every later command.

Build a JSONL input file

One request per line. Each line needs a custom_id you choose (echoed back on the result), and a body that is exactly what you would POST to /v1/chat/completions online.

cat > requests.jsonl <<'EOF'
{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openrelay/gpt-oss-120b", "messages": [{"role": "user", "content": "Name three primary colors."}]}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openrelay/gpt-oss-120b", "messages": [{"role": "user", "content": "Name three prime numbers."}]}}
EOF

Upload the file and create the batch

One command uploads the file and submits the batch (completion_window defaults to 24h):

orl batches create --input-file requests.jsonl --endpoint /v1/chat/completions

uploaded requests.jsonl as file-a1b2c3d4e5f6a7b8c9d0e1f2
{
  "id": "batch_9f8e7d6c5b4a",
  "status": "validating",
  ...
}

To upload first and submit separately, use orl files upload requests.jsonl and pass the returned id with --input-file-id.

Upload with purpose=batch; the response's id is your input_file_id:

curl https://inference.openrelay.inc/v1/files \
  -H "Authorization: Bearer $OPENRELAY_API_KEY" \
  -F "purpose=batch" \
  -F "file=@requests.jsonl"

{
  "id": "file-a1b2c3d4e5f6a7b8c9d0e1f2",
  "object": "file",
  "bytes": 412,
  "created_at": 1752460800,
  "expires_at": 1755052800,
  "filename": "file-a1b2c3d4e5f6a7b8c9d0e1f2.jsonl",
  "purpose": "batch"
}

Then create the batch. completion_window must be "24h":

curl https://inference.openrelay.inc/v1/batches \
  -H "Authorization: Bearer $OPENRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-a1b2c3d4e5f6a7b8c9d0e1f2",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

{
  "id": "batch_9f8e7d6c5b4a",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "input_file_id": "file-a1b2c3d4e5f6a7b8c9d0e1f2",
  "completion_window": "24h",
  "status": "validating",
  "created_at": 1752460801,
  "expires_at": 1752547201,
  "request_counts": { "total": 0, "completed": 0, "failed": 0 }
}

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.openrelay.inc/v1",
    api_key=os.environ["OPENRELAY_API_KEY"],
)

batch_input = client.files.create(
    file=open("requests.jsonl", "rb"),
    purpose="batch",
)

batch = client.batches.create(
    input_file_id=batch_input.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)
print(batch.id, batch.status)  # batch_9f8e7d6c5b4a validating

For small jobs you can skip the upload and pass the requests inline with a requests array instead of input_file_id. See Batches.

Poll until it finishes

Fetch the batch until status is terminal (completed, failed, expired, or cancelled). request_counts fills in as work progresses.

orl batches get batch_9f8e7d6c5b4a

orl batches list shows your recent batches at a glance, and orl batches cancel <id> stops one you no longer need.

curl https://inference.openrelay.inc/v1/batches/batch_9f8e7d6c5b4a \
  -H "Authorization: Bearer $OPENRELAY_API_KEY"

import time

terminal = {"completed", "failed", "expired", "cancelled"}
while batch.status not in terminal:
    time.sleep(30)
    batch = client.batches.retrieve(batch.id)
    print(batch.status, batch.request_counts)

{
  "id": "batch_9f8e7d6c5b4a",
  "status": "completed",
  "output_file_id": "file-batch_9f8e7d6c5b4a-output",
  "request_counts": { "total": 2, "completed": 2, "failed": 0 },
  "usage": { "input_tokens": 34, "output_tokens": 58, "cost_nano_usd": 41000 }
}

Download the results

The output_file_id holds the successful responses; if any requests failed, an error_file_id holds those. Both are JSONL, one line per request, joined to your inputs by custom_id.

orl files content get file-batch_9f8e7d6c5b4a-output -o results.jsonl

curl https://inference.openrelay.inc/v1/files/file-batch_9f8e7d6c5b4a-output/content \
  -H "Authorization: Bearer $OPENRELAY_API_KEY" \
  -o results.jsonl

if batch.output_file_id:
    client.files.content(batch.output_file_id).write_to_file("results.jsonl")
if batch.error_file_id:
    client.files.content(batch.error_file_id).write_to_file("errors.jsonl")

Each line pairs your custom_id with the response. See Results for the exact shape.

Next steps

Files: presigned uploads for large files, and reading file metadata and content.
Batches: the full batch object, inline requests, listing, and cancellation.
Results: the output and error file formats.
CLI reference: install orl and authenticate once with orl auth login.

Quickstart

Build a JSONL input file

Upload the file and create the batch

Poll until it finishes

Download the results

Next steps

On this page