How much does the NVIDIA H200 NVL cost per hour to rent?

On Runcrate the NVIDIA H200 NVL starts at $4.63/hr per GPU on demand today, up to $5.26/hr depending on region and configuration. Billed by the second on most GPUs (a few bill by the minute), with no commitment. For a cluster of them, reserve one: we verify your company and your need, then take your exact spec to GPU owners that fit, and you see their offers side by side.

Can I rent the NVIDIA H200 NVL on demand right now?

Yes. The NVIDIA H200 NVL is on demand in 2 regions today, billed by the second on most GPUs (a few bill by the minute) with no contract.

H200 NVL vs H200 SXM: what's the difference?

The same Hopper GPU with the same 141 GB of HBM3e at 4.8 TB/s. The NVL is a PCIe card at 600 W with 835 TFLOPS of dense FP16 tensor throughput; the SXM module runs at 700 W with 989 (NVIDIA spec). On H200 SXM, full 8-GPU machines have NVLink between all their GPUs. On H200 NVL, GPUs are linked in groups of four; see each offer.

Can I run Llama 405B on an H200 NVL?

Not on one: Llama 405B is 405 GB of weights in FP8, above 141 GB. In FP8 it fits across 4 GPUs (564 GB), more comfortably on an 8-GPU machine. On one H200 NVL, Llama 70B fits in FP8 (70 GB) with room for KV cache. Mistral Large 2 is 123 GB in FP8, so 2 GPUs is the practical setup.

NVIDIA · Hopper · 2024

NVIDIA H200 NVL. 141 GB HBM3e, 835 TFLOPS FP16 tensor.

Name: NVIDIA H200 NVL
Brand: NVIDIA
Price: 4.63 USD
Availability: InStock

The H200's 141 GB of HBM3e on a PCIe card, for LLM inference that outgrows 80 GB.

Launch H200 NVL Reserve a cluster

H200 NVL on demand

from/hr per GPU

Availability: Available now · 2 regions
By region and size: $4.63 to $5.26/hr
Prepaid: 1 mo $4.40/hr3 mo $4.26/hr6 mo $4.13/hr12 mo $3.94/hr
Billing: By the second on most GPUs, a few by the minute
8+ GPUs or a month: Launch any size yourself, or prepay a machine for 1 to 12 months at a discount. Custom quotes on request.

The card

141 GB of HBM3e. At 4.8 TB/s.

Memory: 141 GB; HBM3e
Bandwidth: 4.8 TB/s; Memory to compute
FP16 tensor: 835; TFLOPS
Per node: 8 GPUs; NVLink bridge (groups of 4), 600 W each

By machine size

1x: Single GPU.
2x: 2 GPUs. Usually linked by NVLink bridge; on some machines they connect over PCIe instead.
4x: 4 GPUs. Up to all four in one NVLink bridge group; on some machines they are split across two groups and part of the traffic goes over PCIe.
8x: 8 GPUs in two NVLink bridge groups of four. GPUs talk over NVLink within a group and over PCIe between groups.

Workloads

What teams run on the H200 NVL. Each one opens the launch form, sized.

Llama 70B InferenceFP8 weights (70 GB) leave about 70 GB for KV cacheLaunch it Fine-tune Llama 70BQLoRA on 1 GPU (4-bit weights about 35 GB)Launch it Llama 8B ServingBF16 weights (16 GB) leave about 125 GB for KV cacheLaunch it Stable Diffusion BatchLarger batches than H100: 141 GB vs 80 GBLaunch it

Full specs

NVIDIA H200 NVL specifications.

Memory

Memory: 141 GB
Type: HBM3e
Bandwidth: 4.8 TB/s

Compute

FP32: 60 TFLOPS
FP16 tensor: 835 TFLOPS
INT8: 1670 TOPS

Power and form

TDP: 600 W
Form factor: PCIe
Architecture: Hopper
Released: 2024

Node

GPUs per node: 8
Interconnect: NVLink bridge (groups of 4)
Maker: NVIDIA

Alternatives

Alternatives to the H200 NVL. Same account, different price and performance.

GPU	Memory · FP16 tensor	Stands out	Price gap	From
NVIDIA H200 NVLThis GPU	141 GB HBM3e · 835 TFLOPS			$4.63/hr
NVIDIA H200	141 GB HBM3e · 989 TFLOPS		+11%	$5.13/hr
NVIDIA H100	80 GB HBM3 · up to 989 TFLOPS	-61 GB memory	-23%	$3.55/hr
NVIDIA GH200	96 GB HBM3 · 990 TFLOPS	-45 GB memory		Not in stock right now
NVIDIA L40S	48 GB GDDR6 · 362 TFLOPS	-93 GB memory	-88%	$0.54/hr

Questions

Renting the H200 NVL. What buyers ask first.

Other GPUs