OpenRelay API Documentation

Use the official OpenAI Python and Node libraries with OpenRelay by setting base_url and api_key.

Because the Inference API is OpenAI-compatible, you can use the official OpenAI SDKs unchanged — just point them at OpenRelay's base URL and pass your vl_ API key.

Set base_url to https://inference.openrelay.inc/v1 (with the /v1 suffix — the SDKs append paths like /chat/completions to it) and api_key to your OpenRelay key.

Python

Install the openai package, then construct the client with OpenRelay's base URL and key:

pip install openai

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.openrelay.inc/v1",
    api_key="vl_your_api_key",  # or os.environ["OPENRELAY_API_KEY"]
)

resp = client.chat.completions.create(
    model="openrelay/gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(resp.choices[0].message.content)
print(resp.usage)

Streaming (Python)

stream = client.chat.completions.create(
    model="openrelay/gpt-oss-120b",
    messages=[{"role": "user", "content": "Stream a short poem."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Node / TypeScript

Install the openai package, then construct the client the same way:

npm install openai

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://inference.openrelay.inc/v1',
  apiKey: process.env.OPENRELAY_API_KEY, // 'vl_...'
});

const resp = await client.chat.completions.create({
  model: 'openrelay/gpt-oss-120b',
  messages: [{ role: 'user', content: 'Hello!' }],
});

console.log(resp.choices[0].message.content);
console.log(resp.usage);

Streaming (Node)

const stream = await client.chat.completions.create({
  model: 'openrelay/gpt-oss-120b',
  messages: [{ role: 'user', content: 'Stream a short poem.' }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}

Environment-variable convention

Most OpenAI-compatible tools read OPENAI_API_KEY and OPENAI_BASE_URL from the environment. To point such a tool at OpenRelay without code changes:

export OPENAI_API_KEY="vl_your_api_key"
export OPENAI_BASE_URL="https://inference.openrelay.inc/v1"

import os
from openai import OpenAI

os.environ["OPENAI_API_KEY"] = "vl_your_api_key"
os.environ["OPENAI_BASE_URL"] = "https://inference.openrelay.inc/v1"

client = OpenAI()  # picks up both from the environment

process.env.OPENAI_API_KEY = 'vl_your_api_key';
process.env.OPENAI_BASE_URL = 'https://inference.openrelay.inc/v1';

import OpenAI from 'openai';
const client = new OpenAI(); // picks up both from the environment

Use the OpenRelay model ids (e.g. openrelay/gpt-oss-120b) in the model field.

Using OpenAI SDKs

Python

Streaming (Python)

Node / TypeScript

Streaming (Node)

Environment-variable convention

On this page