Skip to main content
A cluster is two or more GPU machines (its nodes) that you create, connect to, pay for and terminate as one thing. Every machine has the same GPUs, they share a private network and a /shared folder, and they reach each other over SSH by name (node-0, node-1, …). Node 0 is the head. You pick the software that runs across the machines: See Software for what each one installs and when it counts as ready.

​
Bought by the machine

A cluster is priced by the machine, never by the GPU. You pick a GPU and how many machines, and Runcrate shows the machines that can start now: each offer is a location, how the machines are joined, the kind of machine and the GPUs each one comes with (1, 2, 4 or 8), with the price per hour for the whole cluster and the price per machine under it. The whole cluster is the machine price times the number of machines.

​
How a cluster runs

1

Pick the machines

Choose a GPU and how many machines (2 to 64), then one of the offers for that size.
2

Create

Every machine is reserved at once. Status is provisioning while they boot and join the private network.
3

Every machine is up

When every machine has joined, billing starts for all of them at the same moment. With software picked, status is installing while it installs; with None the cluster is ready straight away.
4

Ready

The software checks itself on every node (for Slurm, every node idle; for Kubernetes, every node Ready with its GPUs schedulable; for Ray, every node in ray status), then the cluster is ready and you get an email.
5

Terminate

Terminating stops every machine together, and billing for the whole cluster ends.

​
All or nothing

A cluster starts whole or not at all. If any machine fails to start, or the machines cannot all reach each other within 25 minutes (longer on machines that are known to boot slowly), Runcrate releases every machine it reserved and the cluster is failed. Nothing is billed for a cluster that never had every machine up, and nothing is started again without you: the cluster page shows the offers available now, and you choose. It ends whole too. One node of a cluster cannot be terminated on its own from the Instances page or the API: you terminate the cluster. When your balance runs out, or the environment reaches its monthly budget, the whole cluster is stopped. If the software fails to install after every machine is up, the cluster is degraded: the machines stay up and reachable over SSH, and you choose whether to retry the install or terminate.

​
Billing

  • Billing starts when every machine has joined, from the same moment for every machine, and runs until the cluster is terminated. Nothing is billed while the machines are still starting.
  • Each machine is billed at its hourly price for the time it runs, the same way an instance is. The whole cluster costs the machine price times the number of machines, per hour.
  • A worker added to a running cluster is billed from the moment it is up.
  • Your balance must cover at least one hour of the whole cluster, and never less than $10, to create one.
  • There are no prepaid or reserved terms on self-serve clusters.
  • On the Billing page a cluster is one line, Cluster usage and its name, which opens to a row per machine.
Clusters have no maximum runtime or idle stop. Terminate the cluster when the work is done.

​
How the machines are joined

Self-serve clusters run on Ethernet: the machines’ own network, with an encrypted private network on top (10.77.0.0/24). Some sites give their machines no public address of their own; there the private network runs over the site’s network, and you reach each machine over SSH through a Runcrate gateway, the same as on any other cluster. The encrypted network moves about 1 to 2 Gbit/s between machines. That suits independent jobs on every node, Slurm job arrays, Ray tasks, Kubernetes workloads and LoRA fine-tuning. It is not built for training a full model across machines, where GPUs on different machines exchange gradients every step. Run the NCCL test after creating a cluster to see the bandwidth you actually get. Clusters on a fast InfiniBand or RoCE network are coming; ask us for one with Request a quote. Each offer also names its kind of machine: virtual machine, bare metal or container. Kubernetes is offered on virtual machines and bare metal.

​
Community hosts

Offers are on datacenter hosts unless you ask for community hosts as well: Include community hosts in the dashboard, include_community in the API and MCP, or --include-community in the CLI. Community hosts are machines run by independent operators; keep secrets you would not give them off those clusters. A volume cannot be attached to a cluster on community hosts.

​
Shared storage

On virtual machines and bare metal every node mounts /shared, a folder exported from the head over the private network, with the hostfile in it. Attach a volume when you create the cluster and its files are synced into /shared/volume on the head, and back to the volume, so they outlive the cluster. Container machines share no folder: each node keeps its own disk, and a volume shows on the head only. Everything else on the machines, /shared included, is deleted when the cluster is terminated.

​
Request a quote

For reserved capacity, very large clusters or a fast InfiniBand or RoCE network, use Request a quote on the Clusters page and the team gets back to you. Self-serve clusters and quoted clusters live side by side.

​
Next steps

Create a cluster

From the dashboard, your agent, the CLI or the API.

Software

Slurm, Kubernetes and Ray: what is installed and how to use it.

Connect and run

SSH, the hostfile, torchrun, mpirun, DeepSpeed and the NCCL test.