GPU comparison
H100 vs RTX 4080.
Pick the H100 for memory: 80 GB vs 16 GB, with up to 4.6x the memory bandwidth. A 70B model in 8-bit weights (70 GB) fits on one H100, not on one RTX 4080. For models under 16 GB pick by price.
Rent either on Runcrate
- H100from $3.14/hr
$0.039 per GB/hr
- RTX 4080Not in stock right now
Live hourly price, then per GB of memory.
Which is better for
Which to pick for each job. Memory fit, from the datasheets.
Hold a 70B model in 8-bit weights
70 GB of 8-bit weights
H100: 1 GPU, 10 GB spare
RTX 4080: 5 GPUs, more than one 2x node
H100. Fits on one GPU.
Serve an 8B model in 16-bit weights
16 GB of 16-bit weights
H100: 1 GPU, 64 GB spare
RTX 4080: 2 GPUs (32 GB)
H100. Fits on one GPU.
Fine-tune a 70B model with QLoRA
About 46 GB with 4-bit weights
H100: 1 GPU, 34 GB spare
RTX 4080: 3 GPUs, more than one 2x node
H100. Fits on one GPU.
Compare another pair
Hopper
NVIDIA H100
- Memory
- 80 GB HBM3
- Bandwidth
- up to 3.4 TB/s
Ada Lovelace
NVIDIA RTX 4080
- Memory
- 16 GB GDDR6X
- Bandwidth
- 736 GB/s
At a glance
Spec by spec. The longer bar is the better number.
Full specs
The complete spec sheet.
| Spec | NVIDIA H100 | NVIDIA RTX 4080 |
|---|---|---|
| Architecture | Hopper | Ada Lovelace |
| Memory | 80 GB | 16 GB |
| Memory type | HBM3 | GDDR6X |
| Memory bandwidth | up to 3.4 TB/s | 736 GB/s |
| FP32 | up to 67 TFLOPS | 48.7 TFLOPS |
| FP16 | up to 989 TFLOPS, tensor | 97.5 TFLOPS, non-tensor |
| INT8 | up to 1979 TOPS | 390 TOPS |
| TDP | up to 700 W | 320 W |
| GPUs per node | 8 | 2 |
| Hourly price (live) | from $3.14/hr | Not in stock right now |
Questions
H100 or RTX 4080. What buyers ask first.
Other comparisons