Agents
What an async request can do for an agent today. Tool calls come back for your own loop; OpenRelay-run tool loops and workflows are not available in Async.
Async is in beta
Shapes and limits can still change before general availability.
Each line of an async job is one model call, answered the way the Sync endpoint would answer it.
What works
toolsandtool_choicein a line's body are passed to the model. If it decides to call a tool, the result'sresponse.bodycarries the tool call, as it would on a Sync response.- Your own loop. Run the tool yourself, then add the next turn, with the tool result in its messages, to a new job. For many agent runs in parallel, put one line per run in a job and make the next job from the results of the last.
What does not
openrelay.tool_config (OpenRelay runs the tool loop) and openrelay.workflow
(your callback decides each step) work in the Batch API but are not
supported in an async job. A line that carries either is rejected when the
job is created with invalid_body, is never run and is never billed. Use the
Batch API for those today.
Streaming is not supported either: a line with "stream": true is rejected
with streaming_unsupported.