Benchmarks
GPU startup time. Timed from the API call.
Seconds from the API call to the first CUDA matmul on five GPU types, and how much of a 7B model start is the download.
Measured
186 sto the first CUDA op
L40S, from the create call to a 4096 x 4096 matmul.
On this page
Run 2026-10-26
Launch to first CUDA matmul. Five GPU types.
The L40S was doing math 186 s after the create call returned; the RTX A6000 took 707 s. Billing starts when the machine reports running, which is why each run was billed a cent or two.
Hardware
- VMs, Ubuntu image, one GPU each, launched within 4 seconds of each other at 09:02 UTC
Software
- Install uv, create a venv, uv pip install torch --torch-backend=cu128, then multiply two 4096x4096 matrices on the GPU and synchronize
- Python 3.12
Method
- Seconds since the create call returned: API running (status running with an IP), first SSH (retried every 2 s), first CUDA op
- Kernel boot from uptime -s, ssh.service from systemctl show ssh -p ActiveEnterTimestamp
- Every 10 seconds; a first batch polled every 2 s hit the rate limit and was discarded
What stood out
- Over the day, two of 18 create calls came back within 7 seconds with an error, and one type was gone between listing it and launching it. Both failures were fast and cost nothing.
- A container started 4 seconds after the create call by its own clock, in an earlier batch.
Keep in mind
- Startup depends on what is in stock at that moment; treat it as a sample, not a promise.
- "First CUDA op" includes downloading a 900 MB torch wheel. A pre-built image or a volume with a cached venv removes most of it.
Run 2026-10-29
Hugging Face download vs local cache. A 7B model on an L40S.
From a fresh download the first token came 48 to 55 s after the Python process started; with the weights already on local disk, after 18 to 19 s. The download was 30 s of that, and reading 15 GB from disk into GPU memory took 14 s either way.
Hardware
- One L40S 48 GB, 12 vCPUs, 70 GB RAM
Software
- Qwen/Qwen2.5-7B-Instruct, bf16 safetensors, 15.24 GB on disk (4 shards)
- transformers 4.57.1, huggingface-hub 0.36.2 with hf-xet 1.7.1 (the default download backend), Python 3.12
Method
- One fresh process per measurement: Python imports, snapshot_download, from_pretrained to the GPU in bf16, one generated token
- Two fresh downloads with the defaults, one with HF_XET_HIGH_PERFORMANCE=1, one from local disk after evicting the page cache, two from local disk right after a previous run
Price per hour
| Start | Python imports (s) | Download (s) | Load to GPU (s) | First token (s) | Total (s) | Cost of the wait, run day | Cost of the wait, today |
|---|---|---|---|---|---|---|---|
| Fresh download, defaults (run 1) | 10.2 | 29.6 | 14.0 | 0.57 | 54.5 | $0.0082 | $0.0082 |
| Fresh download, defaults (run 2) | 3.7 | 29.6 | 13.7 | 0.54 | 47.6 | $0.0071 | $0.0071 |
| Fresh download, HF_XET_HIGH_PERFORMANCE=1 | 4.0 | 32.5 | 14.1 | 0.50 | 51.1 | $0.0077 | $0.0077 |
| Local disk cache, page cache evicted | 3.7 | 0.1 | 13.9 | 0.54 | 18.3 | $0.0027 | $0.0027 |
| Local disk cache, in page cache (run 1) | 3.8 | 0.1 | 14.7 | 0.56 | 19.1 | $0.0029 | $0.0029 |
| Local disk cache, in page cache (run 2) | 3.8 | 0.1 | 14.9 | 0.53 | 19.4 | $0.0029 | $0.0029 |
What stood out
- The download was 60% of a fresh start: 29.6 s in both default runs for 15.24 GB, about 515 MB/s.
- HF_XET_HIGH_PERFORMANCE=1 did not help here (32.5 s against 29.6 s).
- Loading to the GPU took 14 s every time, cached or not: 15.24 GB at about 1.1 GB/s, bound by disk reads.
Keep in mind
- One machine, one day, one model. Network speed to the Hub varies by time of day; disk speed varies by machine.
- transformers from_pretrained only. vLLM and other servers add their own startup work on top of the load.
- The page-cache eviction can't be proven, but the load time did not move either way.
Costs
How the costs are worked out. Run day and today, side by side.
Run day
The cost the write-up printed: the hourly price the machine was launched at that day, against what it measured. Prices move daily, so each run-day figure carries its date. Every run was a single run.
Today
The run-day figure moved by today's price: the "from" price per GPU-hour shown on /pricing, over the run-day rate per GPU-hour. A GPU that is not in stock right now reads "Not in stock" rather than a stale price. Prices read 2026-11-04 04:00 UTC.
GPUs
The GPUs on this page. From price per GPU-hour, today.
Benchmarks