Overview
Batch inference jobs: submit a JSONL file of requests, poll status, and download results. A batch has 24 hours to finish; standard batches bill at a discount. Served by the inference endpoint (inference.openrelay.inc). Batch access is enabled per organization; without it these endpoints return 404.
Batches
Batch inference jobs: submit a JSONL file of requests, poll status, and download results. A batch has 24 hours to finish; standard batches bill at a discount. Served by the inference endpoint (inference.openrelay.inc). Batch access is enabled per organization; without it these endpoints return 404.
Answer typed questions with a classifier POST
Answers typed questions about a text or JSON input with a classifier: a choice, a score or a yes/no, with probabilities. Bills input tokens only. Each answer carries its probabilities and a confidence; nothing is generated. Served by openrelay/laya; check a model's supported_apis on GET /v1/models for decisions. Online only: the Batch API does not accept /v1/decisions. Requires a funded balance. Any field this schema does not list is rejected with 400 unknown_field. stream is dropped: a decision is always a single JSON response. Limits: 1 to 64 questions per request, 10 MiB request body.
Get the Batch API's current limits GET
Returns the limits a batch submission runs against today: the per-batch minimum fee and the metadata caps that create and update both enforce. The minimum fee is 0 when none is configured. orl estimate and orl batches create read this instead of assuming fixed constants, so a limit change on the server reaches them without a CLI release.