Benchmarks

A100 vs H100 vs H200. One script, six GPUs.

Name: One script on six GPUs: Six GPUs, one night
Creator: Runcrate
Published: 2026-11-04

The same benchmark script on six rented GPUs in one night: dense matmul, memory copy, nanoGPT, a LoRA fine-tune and SDXL, with what each run cost.

Launch an H200 All benchmarks

Measured

718.46bf16 TFLOPS

H200, dense bf16 matmul at 8192.

Run 2026-10-25, single runSee the table

Run 2026-10-25

One script on six GPUs. Six GPUs, one night.

Dense bf16 matmul at 8192 ran at 718.46 TFLOPS on the H200, 537.94 on the H100 and 251.44 on the A100 80GB. The same 120-step LoRA run trained in 40.2 s on the H200, 55.55 s on the H100 and 78.11 s on the A100 80GB.

Single run: one job per GPU type, on the night of 2026-10-24/25 (UTC).

Hardware

Jobs: One Runcrate job per GPU type, template ubuntu-train, max runtime 30 min (40 min on the RTX A4000), four benchmarks in order, each in its own subprocess

Software

Framework: torch 2.8.0+cu128, CUDA 12.8 on all six machines
Python: Python 3.10

Method

Matmul: Square matmuls of 4096 and 8192 per data type, 3 warm-up calls, then 20 (4096) or 10 (8192) timed with CUDA events
Memory copy: A 512 MB device-to-device copy, read plus write
nanoGPT: karpathy/nanoGPT train_shakespeare_char, --max_iters=2000 --eval_interval=250 --eval_iters=100, torch.compile off
LoRA: Qwen/Qwen2.5-1.5B-Instruct in bf16, LoRA r=16 on all linear layers, gradient checkpointing, micro-batch 4 x 512 tokens, 2 accumulation steps (4,096 tokens per step), 120 steps
SDXL: SDXL base 1.0, fp16, 1024x1024, 30 steps, fixed seeds; one warm-up image, 8 single images, then 2 batches of 4 on cards with at least 24 GB

Price per hour

H200$4.63 on 2026-10-25, from $4.63/hr per GPU today
H100$3.55 on 2026-10-25, from $3.87/hr per GPU on a 4-GPU machine today
A100 80GB$1.73 on 2026-10-25, from $1.73/hr per GPU today
L40S$1.14 on 2026-10-25, from $0.54/hr per GPU on a 2-GPU machine today
RTX A6000$0.74 on 2026-10-25, from $0.57/hr per GPU today
RTX A4000$1.04 on 2026-10-25, from $1.04/hr per GPU today

Dense bf16 matmul at 8192, TFLOPSbf16 TFLOPS

Measured on 2026-10-25, from the table below.

The six machines

GPU	Card	VRAM (GiB)	Driver	$/hour, run day
H200	H200 NVL	139.8	615.71.09	4.63
H100	H100 PCIe	79.2	570.195.03	3.55
A100 80GB	A100 80GB PCIe	79.3	570.195.03	1.73
L40S	L40S	44.4	580.126.09	1.14
RTX A6000	RTX A6000	47.4	580.126.09	0.74
RTX A4000	RTX A4000	15.7	535.54.03	1.04

Dense matmul TFLOPS by data type and matrix size, and device memory copy

GPU	bf16, 4096	bf16, 8192	fp16, 8192	tf32, 8192	fp32, 8192	Memory copy (GB/s)
H200	722.67	718.46	680.89	371.62	46.43	4060.5
H100	501.41	537.94	498.23	257.38	38.76	1822.9
A100 80GB	259.56	251.44	274.75	106.53	18.96	1717.6
L40S	271.37	215.08	227.48	96.85	41.73	651.6
RTX A6000	128.41	112.82	108.02	59.45	24.13	682.6
RTX A4000	72.58	59.6	61.32	31.37	12.24	391.0

nanoGPT, Shakespeare characters, 2,000 iterations

GPU	ms per iteration (median)	2,000 iterations (s)	Final val loss	$ compute cost, run day	$ compute cost, today
H200	5.62	16.2	1.4572	0.02083	0.02083
H100	8.23	23.8	1.4584	0.02347	0.02559
A100 80GB	11.97	34.4	1.4604	0.01653	0.01653
L40S	14.46	38.5	1.4562	0.01219	0.00577
RTX A6000	23.96	67.3	1.4571	0.01383	0.01065
RTX A4000	43.79	114.3	1.4648	0.03302	0.03302

Run day: the price on 2026-10-25, as the run paid it.

Today: the run-day figure times today's price per GPU-hour over the run-day price per GPU-hour. Today's price is the "from" price per GPU-hour shown on /pricing, which can be on a machine with more GPUs than the run used (the list of GPUs below names it). Prices read 2026-11-04 04:00 UTC.

LoRA fine-tune of Qwen2.5-1.5B-Instruct, 120 steps of 4,096 tokens

GPU	Tokens/s	Train (s)	Peak memory (GiB)	Eval loss before	Eval loss after	$ compute cost, run day	$ compute cost, today
H200	12950	40.2	6.87	1.9746	1.3259	0.0517	0.0517
H100	9215	55.55	6.87	1.9746	1.3175	0.05478	0.05972
A100 80GB	6552	78.11	6.82	1.9744	1.3351	0.03754	0.03754
L40S	5977	85.26	6.82	1.9741	1.3181	0.027	0.013
RTX A6000	3340	151.5	6.82	1.9746	1.3384	0.03114	0.02399
RTX A4000	2050	240.82	6.82	1.9749	1.3256	0.06957	0.06957

Run day: the price on 2026-10-25, as the run paid it.

SDXL base 1.0, fp16, 1024 x 1024, 30 steps

GPU	s per image, batch 1	s per image, batch 4	Images per hour	$ per 1,000, run day	$ per 1,000, today
H200	1.697	1.407	2559	1.81	1.81
H100	2.344	2.046	1760	2.018	2.200
A100 80GB	3.628	3.408	1056	1.638	1.638
L40S	3.836	3.689	976	1.168	0.553
RTX A6000	6.568	5.975	603	1.228	0.946
RTX A4000	11.89	not measured	303	3.435	3.435

The RTX A4000 has 16 GB, so it ran batch 1 only.

Run day: the price on 2026-10-25, as the run paid it.

What stood out

Device memory copy reached 4060.5 GB/s on the H200, against 1822.9 on the H100 and 1717.6 on the A100 80GB.
SDXL took 1.697 s per image, one image at a time, on the H200, 2.344 s on the H100 and 3.628 s on the A100 80GB.
Every card reached a nanoGPT validation loss between 1.4562 and 1.4648 after 2,000 iterations: the same training, at different speeds.

Keep in mind

One job per GPU type, on one night. Treat it as a sample.
The cards are not all the same form factor: the H100 and A100 80GB were PCIe cards and the H200 an NVL card, each machine with its own driver (first table).
Compute cost is measured seconds times the job's hourly rate, not the billed total of the job.

Costs

How the costs are worked out. Run day and today, side by side.

Run day

The cost the write-up printed: the hourly price the machine was launched at that day, against what it measured. Prices move daily, so each run-day figure carries its date. Every run was a single run.

Today

The run-day figure moved by today's price: the "from" price per GPU-hour shown on /pricing, over the run-day rate per GPU-hour. A GPU that is not in stock right now reads "Not in stock" rather than a stale price. Prices read 2026-11-04 04:00 UTC.

GPUs

The GPUs on this page. From price per GPU-hour, today.

H200from $4.63/hr per GPU
H100from $3.87/hr per GPU on a 4-GPU machine
A100 80GBfrom $1.73/hr per GPU
L40Sfrom $0.54/hr per GPU on a 2-GPU machine
RTX A6000from $0.57/hr per GPU
RTX A4000from $1.04/hr per GPU

Benchmarks

More benchmarks.

LLM inferenceCost per GPU.$0.12 per 1M output tokens. RTX A6000, vLLM, 64 concurrent requests, at the run-day price, run 2026-10-26, single run.Speech to textAudio hours per dollar.315x real time. RTX 5090, faster-whisper large-v3-turbo, batch 16, run 2026-10-28, single run.Text embeddingsCost per passage.$0.032 per 1M passages. L40S, bge-small, 128-token passages, at the run-day price, run 2026-10-29, single run.Training and fine-tuningMemory, speed, cost.14.5 GiB peak memory. LoRA of Qwen2.5-1.5B on an L40S, micro-batch 4, run 2026-10-28, single run.Image generationImages per dollar.$0.91 per 1,000 images. RTX A6000, compiled, batch 1, at the run-day price, run 2026-10-27, single run.GPU startup timeTimed from the API call.186 s to the first CUDA op. L40S, from the create call to a 4096 x 4096 matmul, run 2026-10-26, single run.