Overview
Generate a model response for a conversation, OpenAI-compatible: the core online inference family, with `orl chat` as its friendly front. Served by the inference endpoint (inference.openrelay.inc).
Chat Completions
Generate a model response for a conversation, OpenAI-compatible: the core online inference family, with orl chat as its friendly front.
Served by the inference endpoint (inference.openrelay.inc).
List GPU capacity reserved to this org GET
Returns the GPU reservations this organization is in the audience of or manages, with node ids, reserved units and terms. Expired and cancelled ones are left out; pending and needs-review reservations are included. With reservations not enabled for the org, the list is empty.
Generate a chat completion POST
Generates a model response for a conversation in the OpenAI Chat Completions shape, from any model whose supported APIs include chat completions. The hand-written `orl chat` is the friendlier front for this operation: model name completion, a readable transcript, and streaming output rendered to the terminal. Requires a funded balance. Set "stream" to true to receive the completion incrementally as Server-Sent Events, ending with a `data: [DONE]` sentinel. The request body is capped at 10 MiB.