Batches
Create a batch inference job
Submit a batch from an uploaded input file (input_file_id) or inline request objects (requests); provide exactly one of the two. The batch completes within the 24h window at a discounted rate; poll it with get, then download results via the output and error file ids.
Authorization
apiKey AuthorizationBearer <token>
OpenRelay API key. Send it as Authorization: Bearer vl_….
In: header
Request Body
application/json
TypeScript Definitions
Use the request body type in TypeScript.
Response Body
application/json
application/json
application/json
application/json
application/json
application/json
application/json
curl -X POST "https://example.com/v1/batches" \ -H "Content-Type: application/json" \ -d '{ "endpoint": "/v1/chat/completions", "completion_window": "24h" }'{
"id": "string",
"object": "batch",
"endpoint": "string",
"input_file_id": "string",
"completion_window": "24h",
"status": "validating",
"model": "string",
"output_file_id": "string",
"error_file_id": "string",
"created_at": 0,
"expires_at": 0,
"completed_at": 0,
"failed_at": 0,
"expired_at": 0,
"cancelled_at": 0,
"request_counts": {
"total": 0,
"completed": 0,
"failed": 0
},
"usage": {
"input_tokens": 0,
"output_tokens": 0,
"cost_nano_usd": 0
},
"metadata": {
"property1": "string",
"property2": "string"
}
}{
"error": {
"message": "string",
"type": "string",
"code": "string"
}
}{
"error": {
"message": "string",
"type": "string",
"code": "string"
}
}{
"error": {
"message": "string",
"type": "string",
"code": "string"
}
}{
"error": {
"message": "string",
"type": "string",
"code": "string"
}
}{
"error": {
"message": "string",
"type": "string",
"code": "string"
}
}{
"error": {
"message": "string",
"type": "string",
"code": "string"
}
}Command line
The orl CLI wraps this endpoint:
orl batches create --completion-window <completion-window> --endpoint <endpoint>