Get a batch by id
Returns one batch with its status, priority and request counts, plus its usage once any is recorded and the output and error file ids once written. Poll it to track progress.
Required permission
Requires the batch.read permission. See Permissions for the roles and API key permissions that include it.
Command line
The orl CLI wraps this endpoint:
orl batches get <id>apiKeyAuthorizationBearer <token>OpenRelay API key. Send it as Authorization: Bearer or_… (legacy vl_… keys still work).
id*stringOK
application/json- response
A batch inference job (OpenAI-compatible).
id*stringBatch id (batch_…).
object*"batch""batch"endpoint*stringThe API family every record in the batch calls.
input_file_id*stringcompletion_window*"24h""24h"priority*stringThe scheduling tier this batch runs and is billed at. Batches created before the tier existed read as standard.
"standard""expedited"status*stringLifecycle state. validating → in_progress → finalizing → completed | failed | expired; cancelling → cancelled. A batch is in_progress while it waits for GPU capacity too; in_progress_at says when its requests started running.
"validating""in_progress""finalizing""completed""failed""expired""cancelling""cancelled"model?stringModel id, settled from the input file during validation.
output_file_id?stringPresent once results are written; fetch its content for successful records.
error_file_id?stringPresent once results are written; fetch its content for failed records.
created_at*integerUnix timestamp (seconds).
int64in_progress_at*integerUnix timestamp (seconds) when the batch's first requests started running. Always present and null until then, so an in_progress batch with a null in_progress_at is still queued for GPU capacity.
int64expires_at?integerWhen the completion window closes (created_at + 24h).
int64completed_at?integerint64failed_at?integerint64expired_at?integerint64cancelled_at?integerint64request_counts*Progress counters, settled as the batch validates and shards complete.
usage?Rolled-up token and cost totals, present once any progress is recorded. cost_nano_usd is the price you pay (batch discount applied), in nano-USD so sub-cent batches stay exact.
metadata?Your key-value pairs, echoed back unchanged.
curl -X GET 'https://inference.openrelay.inc/v1/batches/string'{ "id": "string", "object": "batch", "endpoint": "string", "input_file_id": "string", "completion_window": "24h", "priority": "standard", "status": "validating", "model": "string", "output_file_id": "string", "error_file_id": "string", "created_at": 0, "in_progress_at": 0, "expires_at": 0, "completed_at": 0, "failed_at": 0, "expired_at": 0, "cancelled_at": 0, "request_counts": { "total": 0, "completed": 0, "failed": 0 }, "usage": { "input_tokens": 0, "output_tokens": 0, "cost_nano_usd": 0 }, "metadata": { "property1": "string", "property2": "string" }}Create a batch inference job POST
Submit a batch from an uploaded input file (input_file_id) or inline request objects (requests); provide exactly one of the two. A batch has 24 hours from submission (expires_at) to finish: records not done by then come back with the batch_expired error and the batch ends expired. Standard batches bill at the discounted batch rate. Poll it with get, then download results via the output and error file ids. An API key whose spend budget is used up gets 402 key_budget_exceeded; a batch already running finishes, and its billed usage counts toward the key.
Cancel a running batch POST
Cancels a batch that has not finished. In-flight records finish, unstarted ones are skipped, and partial results are written; completed work is billed. The batch settles to cancelled within a few minutes. A batch already in a terminal state returns 409.