GPU comparison
GH200 vs RTX 4080.
The GH200 is ahead on memory (96 GB vs 16 GB) and speed (10.2x the FP16 throughput and up to 5.4x the memory bandwidth). The RTX 4080 makes sense when your model fits in 16 GB and its hourly price is lower.
Rent either on Runcrate
- GH200Not in stock right now
- RTX 4080Not in stock right now
Which is better for
Which to pick for each job. Memory fit and throughput, from the datasheets.
Hold a 70B model in 8-bit weights
70 GB of 8-bit weights
GH200: 1 GPU, 26 GB spare
RTX 4080: 5 GPUs, more than one 2x node
GH200. Fits on one GPU.
Serve an 8B model in 16-bit weights
16 GB of 16-bit weights
GH200: 1 GPU, 80 GB spare
RTX 4080: 2 GPUs (32 GB)
GH200. Fits on one GPU.
Fine-tune a 70B model with QLoRA
About 46 GB with 4-bit weights
GH200: 1 GPU, 50 GB spare
RTX 4080: 3 GPUs, more than one 2x node
GH200. Fits on one GPU.
Train on a full node
FP16 tensor throughput per node
GH200: 1x, 990 TFLOPS
RTX 4080: 2x, 195 TFLOPS
GH200. 10.2x the FP16 throughput per GPU.
Throughput per watt
FP16 TFLOPS per 100 W of TDP
GH200: Superchip power, not a GPU TDP
RTX 4080: 30 TFLOPS per 100 W
Either. Not comparable.
Compare another pair
Hopper
NVIDIA GH200
- Memory
- 96 GB HBM3
- FP16 tensor
- 990 TFLOPS
- Bandwidth
- up to 4.0 TB/s
Ada Lovelace
NVIDIA RTX 4080
- Memory
- 16 GB GDDR6X
- FP16 tensor
- 97.5 TFLOPS
- Bandwidth
- 736 GB/s
At a glance
Spec by spec. The longer bar is the better number.
Full specs
The complete spec sheet.
| Spec | NVIDIA GH200 | NVIDIA RTX 4080 |
|---|---|---|
| Architecture | Hopper | Ada Lovelace |
| Memory | 96 GB | 16 GB |
| Memory type | HBM3 | GDDR6X |
| Memory bandwidth | up to 4.0 TB/s | 736 GB/s |
| FP32 | 67 TFLOPS | 48.7 TFLOPS |
| FP16 tensor | 990 TFLOPS | 97.5 TFLOPS |
| INT8 | 1979 TOPS | 390 TOPS |
| Power | up to 1000 W per superchip (CPU, GPU and memory) | 320 W per GPU (TDP) |
| Form factor | Superchip | Consumer |
| NVLink | One GPU | No |
| GPUs per node | 1 | 2 |
| Hourly price (live) | Not in stock right now | Not in stock right now |
Questions
GH200 or RTX 4080. What buyers ask first.
Other comparisons