Cancel a running batch
Cancels a batch that has not finished. In-flight records finish, unstarted ones are skipped, and partial results are written; completed work is billed. The batch settles to cancelled within a few minutes. A batch already in a terminal state returns 409.
Required permission
Requires the batch.write permission. See Permissions for the roles and API key permissions that include it.
Command line
The orl CLI wraps this endpoint:
orl batches cancel <id>apiKeyAuthorizationBearer <token>OpenRelay API key. Send it as Authorization: Bearer or_… (legacy vl_… keys still work).
id*stringOK
application/json- response
A batch inference job (OpenAI-compatible).
id*stringBatch id (batch_…).
object*"batch""batch"endpoint*stringThe API family every record in the batch calls.
input_file_id*stringcompletion_window*"24h""24h"priority*stringThe scheduling tier this batch runs and is billed at. Batches created before the tier existed read as standard.
"standard""expedited"status*stringLifecycle state. validating → in_progress → finalizing → completed | failed | expired; cancelling → cancelled. A batch is in_progress while it waits for GPU capacity too; in_progress_at says when its requests started running.
"validating""in_progress""finalizing""completed""failed""expired""cancelling""cancelled"model?stringModel id, settled from the input file during validation.
output_file_id?stringPresent once results are written; fetch its content for successful records.
error_file_id?stringPresent once results are written; fetch its content for failed records.
created_at*integerUnix timestamp (seconds).
int64in_progress_at*integerUnix timestamp (seconds) when the batch's first requests started running. Always present and null until then, so an in_progress batch with a null in_progress_at is still queued for GPU capacity.
int64expires_at?integerWhen the completion window closes (created_at + 24h).
int64completed_at?integerint64failed_at?integerint64expired_at?integerint64cancelled_at?integerint64request_counts*Progress counters, settled as the batch validates and shards complete.
usage?Rolled-up token and cost totals, present once any progress is recorded. cost_nano_usd is the price you pay (batch discount applied), in nano-USD so sub-cent batches stay exact.
metadata?Your key-value pairs, echoed back unchanged.
curl -X POST 'https://inference.openrelay.inc/v1/batches/string/cancel'{ "id": "string", "object": "batch", "endpoint": "string", "input_file_id": "string", "completion_window": "24h", "priority": "standard", "status": "validating", "model": "string", "output_file_id": "string", "error_file_id": "string", "created_at": 0, "in_progress_at": 0, "expires_at": 0, "completed_at": 0, "failed_at": 0, "expired_at": 0, "cancelled_at": 0, "request_counts": { "total": 0, "completed": 0, "failed": 0 }, "usage": { "input_tokens": 0, "output_tokens": 0, "cost_nano_usd": 0 }, "metadata": { "property1": "string", "property2": "string" }}Get a batch by id GET
Returns one batch with its status, priority and request counts, plus its usage once any is recorded and the output and error file ids once written. Poll it to track progress.
Replace a batch's metadata POST
Replace the batch's metadata map with the one you send; the whole map is overwritten, so include every pair you want to keep. Send an empty object to clear it. Metadata is opaque customer data, not lifecycle state, so it is editable at any time including after the batch is terminal. Returns the updated batch.