orl chat
Send one chat completion to a model and print the answer, part of the orl CLI for OpenRelay GPU VMs, pods, batches and billing.
orl chat [prompt]Sends the prompt (the argument, or stdin when it is not a terminal) as a user message to POST /v1/chat/completions on the inference endpoint. --body sends a whole request instead (messages, tools, response_format, ...). Under --json the answer comes with the request id, latency, token usage and the server's cost for the request (cost_nano_usd); against a server that sends no cost, the list price of the usage instead (list_cost_nano_usd).
--max-cost (or ORL_MAX_COST_USD) refuses the call before it is sent when its worst case is over the cap; the worst case needs --max-tokens.
Flags
| Flag | Type | Required | |
|---|---|---|---|
--model | string | no | model id (orl models lists them) |
--system | string | no | a system message before the prompt |
--max-tokens | int | no | cap on output tokens (needed for a cost cap) |
--temperature | float64 | no | sampling temperature |
--body | json | no | a whole chat completion request instead of the prompt (--model overrides its model) |
--max-cost | float64 | no | refuse (exit 8) when the worst-case cost exceeds this many dollars (default: ORL_MAX_COST_USD) |
--timeout | string | no | give up after this long (default 35m; over MCP at most 10m) |
--stream | bool | no | print the answer as it is generated (and report time to first token) |
orl catalog vm-template list
VM template catalog, part of the orl CLI for OpenRelay GPU VMs, pods, batches and billing. Usage, every flag and an example.
orl chat-completions create
Generate a chat completion, part of the orl CLI for OpenRelay GPU VMs, pods, batches and billing. Usage, every flag and an example.