CLICommands
orl batches create
Create a batch inference job, part of the orl CLI for OpenRelay GPU VMs, pods, batches and billing. Usage, every flag and an example.
orl batches createSubmit a batch from a local JSONL file (--input-file, uploaded for you), an uploaded file id, or inline requests. The endpoint defaults to the records' url and the completion window to 24h.
--max-cost (or ORL_MAX_COST_USD) refuses the batch, before anything is uploaded, when its cost bound
exceeds the cap (see orl estimate). --idempotency-key makes a retry return the batch an earlier
attempt created instead of creating another. --wait blocks until the batch finishes and prints it
then; --out and --errors-out also download its results.
Flags
| Flag | Type | Required | |
|---|---|---|---|
--completion-window | 24h | no | Time budget for the batch; only "24h" is supported. |
--endpoint | /v1/chat/completions, /v1/completions, /v1/embeddings | no | The API family every record in the batch calls. |
--input-file-id | string | no | Id of an uploaded JSONL input file (from the file upload). |
--metadata | map[string]string | no | Key-value pairs echoed back on the batch object. Updating metadata later allows at most 32 pairs. |
--model | string | no | Optional display hint shown on the batch immediately; validation settles the authoritative value from the input file. |
--priority | standard, expedited | no | Scheduling tier. standard (the default) runs at the discounted batch rate. expedited is granted a freeing pool slot ahead of any waiting standard batch, grows the fleet to a tighter drain target, and is billed at the online rate with no batch discount; it is available to every organization unless an operator has disabled it for yours, in which case the request is rejected with 400. Neither tier changes the 24-hour expiry. |
--requests | json | no | Inline request objects (max 50,000), each with a custom_id and the request body for the chosen endpoint. Alternative to input_file_id for small batches. |
--input-file | string | no | local JSONL file to upload and use as the batch input |
--max-cost | float64 | no | refuse (exit 8) when the cost bound exceeds this many dollars (default: ORL_MAX_COST_USD) |
--idempotency-key | string | no | return the batch an earlier create with this key made, instead of creating another |
--wait | bool | no | wait until the batch finishes (exit 13 when it fails, expires or is cancelled) |
--timeout | string | no | with --wait, stop waiting after this long (exit 9), e.g. 30m |
--out | string | no | with --wait, write the output records here |
--errors-out | string | no | with --wait, write the failed records here |