Interruptible nodes

Interruptible capacity is a GPU container you rent somewhere else: a spot instance, a GPU marketplace, a cloud container service. The market can take it back at short notice, and it often comes without root, without systemd, and with an image you did not build. You onboard it on your own provider account with one command, run inside the container. Nothing is installed outside one directory, and the command needs no sudo.

Interruptible capacity is NVIDIA-only for now: the agent reads the GPUs through nvidia-smi. It is a separate path from VM and pod nodes: it has its own token class, its own installer, and a different lifecycle (a restarted container is a new node).

RequirementsCopy Anchor Link

You need an org that is an approved provider (see Providers). The container needs:

Not required: root, systemd or any init system, being PID 1, NET_ADMIN or NET_RAW, /dev/net/tun, PCI sysfs or lspci, ip or iptables, a package manager, inbound ports, IPv6, a filesystem that survives a restart, or a cloud metadata server. The node reaches OpenRelay only through outbound connections: it punches through NAT where it can and relays through the lighthouses where it cannot.

Onboard a containerCopy Anchor Link

orl provider provisioning-token create \
  --class interruptible --notice-seconds 30 --max-enrollments 64

curl -fsSL "https://api.openrelay.inc/v1/provider/interruptible/bootstrap?token=otk_..." | sh

script=$(curl -fsSL "https://api.openrelay.inc/v1/provider/interruptible/bootstrap?token=otk_...") \
  && printf '%s\n' "$script" | sh

orl provider node list --class interruptible

No curl or wgetCopy Anchor Link

Fetch the bootstrap URL on any machine that has a downloader and read the script: it names a download URL and a sha256 for each architecture (amd64 and arm64). The download URLs expire an hour after you fetch the script. Download the agent for the container's architecture, check its sha256, copy the binary into the container, and run it there:

chmod +x ./interruptible-agent
./interruptible-agent run --api https://api.openrelay.inc --token otk_...

The agent carries its own CA roots, so this path needs no CA bundle in the image. Add --state-dir <dir> to choose the state directory; without it the agent uses the order in Requirements.

Termination warningsCopy Anchor Link

A market reclaims a container on its own schedule. How it warns you is between you and the market: a metadata endpoint, a signal, a webhook, an email, or nothing. OpenRelay does not watch any market or cloud API. If you can pass a warning on, use one of two doors:

orl provider node offboard <nodeId> --notice-seconds 90
orl provider node offboard <nodeId> --terminate-at 2026-10-01T12:00:00Z

A warning moves the node to draining at once and removes it at the deadline, or earlier if its heartbeats stop first. A deadline can be at most 24 hours ahead (--terminate-at, --at, --notice-seconds and --in alike); a time in the past means now. Sent with no time at all, a warning uses the token's --notice-seconds (30 by default). A deadline only ever moves earlier, so a late or repeated warning cannot extend a node's life.

Sending nothing is supported: the node goes offline when its heartbeats stop and is removed about 13 minutes after it goes quiet.

Stopping the agent itself with SIGTERM or SIGINT counts as a warning with a deadline of now (best effort). In most containers the agent is not PID 1, and a container stop sends SIGTERM only to PID 1, so the agent is killed outright and liveness removes the node instead.

Offboard a nodeCopy Anchor Link

To retire a healthy node now:

orl provider node offboard <nodeId>

With no deadline the node is removed in the same call. The agent sees the removal on its next 30-second poll and exits. orl provider node update <nodeId> --status removed also removes a node at once. Offboard before you stop a container if you can: otherwise the node lingers as offline, then is removed up to 13 minutes later.

Restarts: every run is a new nodeCopy Anchor Link

An Interruptible node has no hardware identity, so every run of the command enrolls a new node with a new ID. Nothing restarts the agent when the container restarts (there is no systemd, and the agent is not PID 1):

• Container restarted: the old node goes offline, then removed about 13 minutes later. Run the command again, by hand or from the start hook, to enroll a new node.
• Command re-run after the node was removed: enrolls a new node if the token is still valid.
• Two containers from the same image: two nodes.

Do not key anything on a node ID across restarts. Every enrollment counts against the token's --max-enrollments, restarts included.

What OpenRelay can accessCopy Anchor Link

• A shell in your container. The agent runs an SSH server inside its own process, reachable only over OpenRelay's private overlay network (port 22022 on the node's overlay address). OpenRelay operators can open a session into the container, as the user the agent runs as, for support and to place workloads on the node. Sessions need a short-lived certificate (24 hours) from OpenRelay's ops SSH authority, issued for that node's ID. There is no password login and no inbound port on your side.
• What the agent reports at enrollment: the driver version and each GPU's model, UUID, PCI address, memory and fabric state (from nvidia-smi -q -x); the container's hostname, CPU model, core count and RAM; and whether it runs under gVisor. Once running, it also reports its SSH host key. The rest of the probe (the NAT mapping, user ID, capabilities, tools) is printed in the container and never sent.
• What it leaves alone: it does not use /etc/machine-id, DMI data or a cloud metadata server, and it watches no market API.

LimitsCopy Anchor Link

TroubleshootingCopy Anchor Link

interruptible-agent probe prints what the agent sees in the container and changes nothing. interruptible-agent status prints the running agent's last state. The log is agent.log in the state directory.