NVIDIA · Hopper · 2023
NVIDIA GH200. 96 GB HBM3, 990 TFLOPS FP16 tensor.
A Grace CPU and a Hopper GPU on one superchip: 96 GB of HBM3, joined to the CPU at 900 GB/s.
Availability
Not in stock right now.
- On demand instead
- H100 $3.55/hr
- Register interest
- Register interest in the NVIDIA GH200
The card
96 GB of HBM3. At up to 4.0 TB/s.
- Memory
- 96 GB
- HBM3
- Bandwidth
- up to 4.0 TB/s
- Memory to compute
- FP16 tensor
- 990
- TFLOPS
- Per node
- 1 GPU
- up to 1000 W per superchip
Workloads
What teams run on the GH200. Sized for this card.
Llama 70B InferenceFP8 weights (70 GB) fit in one GPU's 96 GB
Fine-tune Llama 70BQLoRA on 1 GPU (4-bit weights about 35 GB)
Llama 8B ServingBF16 weights (16 GB) leave about 80 GB for KV cache
Full specs
NVIDIA GH200 specifications.
Memory
- Memory
- 96 GB
- Type
- HBM3
- Bandwidth
- up to 4.0 TB/s
Compute
- FP32
- 67 TFLOPS
- FP16 tensor
- 990 TFLOPS
- INT8
- 1979 TOPS
Power and form
- Superchip power
- up to 1000 W (CPU, GPU and memory)
- Form factor
- Superchip
- Architecture
- Hopper
- Released
- 2023
Node
- GPUs per node
- 1
- Maker
- NVIDIA
Alternatives
Alternatives to the GH200. Same account, different price and performance.
| GPU | Memory · FP16 tensor | Stands out | Price gap | From |
|---|---|---|---|---|
| NVIDIA GH200This GPU | 96 GB HBM3 · 990 TFLOPS | Not in stock right now | ||
| NVIDIA H200 | 141 GB HBM3e · 989 TFLOPS | +45 GB memory | $4.63/hr | |
| NVIDIA H100 | 80 GB HBM3 · up to 989 TFLOPS | $3.55/hr | ||
| AMD MI300X | 192 GB HBM3 · 1307.4 TFLOPS | +96 GB memory | Not in stock right now | |
| NVIDIA L40S | 48 GB GDDR6 · 362 TFLOPS | -48 GB memory | $0.54/hr |
Questions
Renting the GH200. What buyers ask first.
Other GPUs
Browse every GPU by architecture.
NVIDIA Blackwell
NVIDIA Hopper
NVIDIA Ampere
NVIDIA Ada Lovelace
NVIDIA Volta