GPU comparison
GH200 vs RTX 4070 Ti.
The GH200 is ahead on memory (96 GB vs 12 GB) and speed (12.3x the FP16 throughput and up to 7.9x the memory bandwidth). The RTX 4070 Ti makes sense when your model fits in 12 GB and its hourly price is lower.
Rent either on Runcrate
- GH200Not in stock right now
- RTX 4070 TiNot in stock right now
Which is better for
Which to pick for each job. Memory fit and throughput, from the datasheets.
Hold a 70B model in 8-bit weights
70 GB of 8-bit weights
GH200: 1 GPU, 26 GB spare
RTX 4070 Ti: 7 GPUs, more than one 2x node
GH200. Fits on one GPU.
Serve an 8B model in 16-bit weights
16 GB of 16-bit weights
GH200: 1 GPU, 80 GB spare
RTX 4070 Ti: 2 GPUs (24 GB)
GH200. Fits on one GPU.
Fine-tune a 70B model with QLoRA
About 46 GB with 4-bit weights
GH200: 1 GPU, 50 GB spare
RTX 4070 Ti: 4 GPUs, more than one 2x node
GH200. Fits on one GPU.
Train on a full node
FP16 tensor throughput per node
GH200: 1x, 990 TFLOPS
RTX 4070 Ti: 2x, 160 TFLOPS
GH200. 12.3x the FP16 throughput per GPU.
Throughput per watt
FP16 TFLOPS per 100 W of TDP
GH200: Superchip power, not a GPU TDP
RTX 4070 Ti: 28 TFLOPS per 100 W
Either. Not comparable.
Compare another pair
Hopper
NVIDIA GH200
- Memory
- 96 GB HBM3
- FP16 tensor
- 990 TFLOPS
- Bandwidth
- up to 4.0 TB/s
Ada Lovelace
NVIDIA RTX 4070 Ti
- Memory
- 12 GB GDDR6X
- FP16 tensor
- 80.2 TFLOPS
- Bandwidth
- 504 GB/s
At a glance
Spec by spec. The longer bar is the better number.
Full specs
The complete spec sheet.
| Spec | NVIDIA GH200 | NVIDIA RTX 4070 Ti |
|---|---|---|
| Architecture | Hopper | Ada Lovelace |
| Memory | 96 GB | 12 GB |
| Memory type | HBM3 | GDDR6X |
| Memory bandwidth | up to 4.0 TB/s | 504 GB/s |
| FP32 | 67 TFLOPS | 40.1 TFLOPS |
| FP16 tensor | 990 TFLOPS | 80.2 TFLOPS |
| INT8 | 1979 TOPS | 321 TOPS |
| Power | up to 1000 W per superchip (CPU, GPU and memory) | 285 W per GPU (TDP) |
| Form factor | Superchip | Consumer |
| NVLink | One GPU | No |
| GPUs per node | 1 | 2 |
| Hourly price (live) | Not in stock right now | Not in stock right now |
Questions
GH200 or RTX 4070 Ti. What buyers ask first.
Other comparisons