Pods and persistent storage

Bring-your-own-container GPU pods, what survives a stop, restart, crash or node loss, and how the /workspace volume is billed and retained.

A pod runs your container image on one or more whole GPUs of a single node, with SSH access and an HTTPS endpoint. Every pod comes with a persistent volume mounted read-write at /workspace. That volume is what makes a pod safe to stop and start: your data lives there, not in the container filesystem.

What persists

Location	Survives a stop, restart or crash?	Survives delete?
`/workspace` (the volume)	Yes	No
Everything else in the container	No: the container is recreated from the image	No

Put checkpoints, datasets, notebooks and anything you cannot re-download under /workspace. Package installs, files written elsewhere and running processes are gone whenever the container is recreated.

Volume size

Pick the size when you create the pod (volumeGb): the default is 50 GB, the range is 10 to 1024 GB. The size is fixed for the pod's life; there is no resize. The volume lives on the pod's node, is not billed separately and adds nothing to the hourly rate.

The pod page in the dashboard shows how much of the volume is in use. The figure comes from an hourly report by the node, so it can lag by up to an hour.

Creating a pod

In the dashboard, Pods > New pod has a field for the volume. From the CLI or the API, pass --volume-gb / volumeGb (omit it for 50 GB):

orl pods create \
  --image ghcr.io/you/trainer:latest \
  --gpu-model-id <gpuModelId from orl pods capacity list> \
  --gpu-count 1 \
  --volume-gb 200 \
  --ssh-keys "$(cat ~/.ssh/id_ed25519.pub)" \
  --name trainer

The same request as an API call is POST /v1/pods. GET /v1/pods/capacity lists the GPU models with free capacity and their rates.

SSH keys are checked at the org

The SSH gateway authenticates with the SSH keys registered on your organization (the SSH Keys page in the dashboard, orl ssh-keys create, or POST /v1/orgs/{orgId}/ssh-keys/create). A key passed only in sshKeys at create is injected into the container but does not by itself open a session. Register it on the org as well.

Lifecycle

A pod moves through the statuses below. Only running is billed.

Status	Meaning	Billed	GPU held	`/workspace`
`deploying`	Being placed, or being started again	No	Yes	Kept
`running`	Serving	Yes	Yes	Kept
`stopped`	You stopped it; the GPU is released	No	No	Kept
`interrupted`	The node became unreachable; the pod resumes on its own when the node returns	No	Yes	Kept
`terminated`	Deleted	No	No	Deleted

Stop and start

Stop (POST /v1/vms/{id}/stop, orl vms stop) ends the container, releases the GPU and keeps the volume. You are not billed while the pod is stopped.

Start (POST /v1/vms/{id}/restart, orl vms restart on a stopped pod) brings it back on the same node, because that is where the volume is. The GPUs it needs must be free on that node; if another workload has taken them the start is refused with 409 POD_NODE_BUSY and the count of free GPUs, and you can retry later or delete the pod and create a new one elsewhere. A start while the node is offline is refused with 409 POD_NODE_OFFLINE, and a start of an InfiniBand-enabled pod on a node whose agent is still being updated for it with 409 POD_NODE_NO_INFINIBAND.

Restart while running

The same restart call on a running pod recreates the container from the image (a fresh filesystem outside /workspace) and keeps the volume. Billing continues through the restart.

Crashes and node loss

If the container exits or is killed, the pod is recreated automatically on the same node with the volume intact, and the pod stays running.

If the node stops reporting to us, the pod becomes interrupted after a few minutes of grace: billing stops, the volume and the GPU reservation stay with the node, and the pod returns to running by itself once the node is back. You can also stop or delete an interrupted pod.

Delete

Delete (POST /v1/vms/{id}/terminate, orl vms terminate) removes the pod and its volume. There is no undo and no snapshot: copy anything you need out of /workspace first.

Retention: when a volume is deleted without you asking

Two situations delete a pod and its volume automatically. Both send an email first.

Node never came back. An interrupted pod whose node has not returned within 7 days is deleted. If the node is removed from the platform before that, the pod is deleted at that point, because the volume was on the box.
Insufficient balance. When your balance runs out, running pods are stopped and kept with their volumes. If the account is not funded again within 7 days of that stop, those pods and their volumes are deleted. Top up before then and start them again.

A pod you stopped yourself is kept, with its volume, until you delete it.

Billing

A pod bills the GPU rate shown at create for every second it is running, as one usage record per running stint (create to stop, start to interrupt, and so on). Stopped and interrupted time is free. The volume has no charge of its own. See VM billing for how rates, balance and auto-stop work; the same rules apply to pods.

Pods and persistent storage

On this page