Dedicated endpoints

A dedicated endpoint serves a model on capacity reserved for your organization. It has its own private model id and its own price, and only your organization can call it. Dedicated endpoints are set up with our team; to get one, contact us.

Call your endpointCopy Anchor Link

A dedicated endpoint is called through the same inference API as every other model: send its modelId as model.

curl https://inference.openrelay.inc/v1/chat/completions \
  -H "Authorization: Bearer $OPENRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "private/acme-gpt-oss-20b",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Any key in your organization can call it, except a key limited to other dedicated endpoints: a key carrying inference:endpoint:<id> scopes can call only the endpoints it lists and gets 404 for everything else. To give a key access to this endpoint and nothing else, create it with the scope inference:endpoint:<id> (see Legacy scopes). With permissions enabled, list the endpoint in the key's restrictions.endpoints instead; any other model then answers 403 with code model_not_allowed.

Find your endpointsCopy Anchor Link

Your endpoints are listed in the dashboard under Inference > Dedicated, through GET /v1/inference/endpoints, and with the CLI:

orl dedicated-endpoints list
orl dedicated-endpoints get <id>

These reads go to the control plane (api.openrelay.inc) and need Dedicated endpoints: Read (Permissions), or clusters:read on a legacy key, which every read-only key has. A key that only carries inference scopes gets 403. An organization without dedicated inference sees an empty list.

Every change to an endpoint (created, enabled, disabled, repriced) is recorded as an event: GET /v1/inference/endpoints/{id}/events, or orl dedicated-endpoints event list <id>.

Monitor your endpointCopy Anchor Link

The dashboard's Monitoring tab on each endpoint charts its traffic. The same data comes from GET /v1/inference/endpoints/{id}/series:

curl "https://api.openrelay.inc/v1/inference/endpoints/$ENDPOINT_ID/series?range=1h" \
  -H "Authorization: Bearer $OPENRELAY_API_KEY"

orl dedicated-endpoints series get <id> --range 1h

Data appears within a minute of the endpoint serving a request.

RangesCopy Anchor Link

ResponseCopy Anchor Link

Every series shares one time grid. values[i] is the value at start + i * stepSeconds, and every values array is pointCount long.

{
  "model": "private/acme-gpt-oss-20b",
  "range": "1h",
  "start": "2026-09-26T17:30:00Z",
  "end": "2026-09-26T18:30:00Z",
  "stepSeconds": 15,
  "pointCount": 241,
  "metrics": {
    "requests": {
      "unit": "requests_per_second",
      "series": [
        { "key": "success", "values": [3.1, 2.9, null], "window": 5300 },
        { "key": "error", "values": [0.02, 0, null], "window": 88 }
      ]
    },
    "latency": {
      "unit": "seconds",
      "series": [
        { "key": "p50", "values": [2.4, 2.3, null], "window": 2.41 },
        { "key": "p95", "values": [6.1, 6.8, null], "window": 6.79 }
      ]
    }
  }
}

The example is trimmed. Each values array is cut to its first three points (a 1h response has 241), and metrics always carries all five of requests, errors, latency, ttft, and tokens, in the same shape, with all four percentiles in latency and ttft.

• A per-second series (requests, tokens) reads 0 while the endpoint is idle. null there means no data was recorded at that point at all, for example before the endpoint's first request.
• A share (errors) or a percentile (latency, ttft) is null at a point with no requests: there was nothing to measure.
• window is the figure for the whole range, computed from every request in it: a count for a per-second series, a share for a ratio, and the true percentile over the range for a latency series. It is not an average of the points.

What each series meansCopy Anchor Link

Each request is counted once, by what it returned to you:

A request retried on another replica counts once, with its final result. A request you cancel before it completes is not counted. Neither is a request turned away before it is matched to the endpoint: a body that is not valid JSON, names no model, or is over 10 MiB; an invalid API key; a requests-per-minute or tokens-per-minute limit already used up when the request arrived; or a balance below the minimum.