GPU comparison
GH200 vs RTX 3090.
The GH200 is ahead on memory (96 GB vs 24 GB) and speed (13.9x the FP16 throughput and up to 4.3x the memory bandwidth). The RTX 3090 makes sense when your model fits in 24 GB and its hourly price is lower.
Rent either on Runcrate
- GH200Not in stock right now
- RTX 3090Not in stock right now
Which is better for
Which to pick for each job. Memory fit and throughput, from the datasheets.
Hold a 70B model in 8-bit weights
70 GB of 8-bit weights
GH200: 1 GPU, 26 GB spare
RTX 3090: 4 GPUs, more than one 2x node
GH200. Fits on one GPU.
Serve an 8B model in 16-bit weights
16 GB of 16-bit weights
GH200: 1 GPU, 80 GB spare
RTX 3090: 1 GPU, 8 GB spare
GH200. Same GPU count, up to 4.3x the memory bandwidth.
Fine-tune a 70B model with QLoRA
About 46 GB with 4-bit weights
GH200: 1 GPU, 50 GB spare
RTX 3090: 2 GPUs (48 GB)
GH200. Fits on one GPU.
Train on a full node
FP16 tensor throughput per node
GH200: 1x, 990 TFLOPS
RTX 3090: 2x, 142 TFLOPS, NVLink
GH200. 13.9x the FP16 throughput per GPU.
Throughput per watt
FP16 TFLOPS per 100 W of TDP
GH200: Superchip power, not a GPU TDP
RTX 3090: 20 TFLOPS per 100 W
Either. Not comparable.
Compare another pair
Hopper
NVIDIA GH200
- Memory
- 96 GB HBM3
- FP16 tensor
- 990 TFLOPS
- Bandwidth
- up to 4.0 TB/s
Ampere
NVIDIA RTX 3090
- Memory
- 24 GB GDDR6X
- FP16 tensor
- 71 TFLOPS
- Bandwidth
- 936 GB/s
At a glance
Spec by spec. The longer bar is the better number.
Full specs
The complete spec sheet.
| Spec | NVIDIA GH200 | NVIDIA RTX 3090 |
|---|---|---|
| Architecture | Hopper | Ampere |
| Memory | 96 GB | 24 GB |
| Memory type | HBM3 | GDDR6X |
| Memory bandwidth | up to 4.0 TB/s | 936 GB/s |
| FP32 | 67 TFLOPS | 35.6 TFLOPS |
| FP16 tensor | 990 TFLOPS | 71 TFLOPS |
| INT8 | 1979 TOPS | 284 TOPS |
| Power | up to 1000 W per superchip (CPU, GPU and memory) | 350 W per GPU (TDP) |
| Form factor | Superchip | Consumer |
| NVLink | One GPU | Yes |
| GPUs per node | 1 | 2 |
| Hourly price (live) | Not in stock right now | Not in stock right now |
Questions
GH200 or RTX 3090. What buyers ask first.
Other comparisons