Skip to content

GPU comparison

H100 vs RTX 4080.

Pick the H100 for memory: 80 GB vs 16 GB, with up to 4.6x the memory bandwidth. A 70B model in 8-bit weights (70 GB) fits on one H100, not on one RTX 4080. For models under 16 GB pick by price.

Rent either on Runcrate

Live hourly price, then per GB of memory.

Which is better for

Which to pick for each job. Memory fit, from the datasheets.

  • Hold a 70B model in 8-bit weights

    70 GB of 8-bit weights

    H100: 1 GPU, 10 GB spare

    RTX 4080: 5 GPUs, more than one 2x node

    H100. Fits on one GPU.

  • Serve an 8B model in 16-bit weights

    16 GB of 16-bit weights

    H100: 1 GPU, 64 GB spare

    RTX 4080: 2 GPUs (32 GB)

    H100. Fits on one GPU.

  • Fine-tune a 70B model with QLoRA

    About 46 GB with 4-bit weights

    H100: 1 GPU, 34 GB spare

    RTX 4080: 3 GPUs, more than one 2x node

    H100. Fits on one GPU.

Compare another pair

vs

Hopper

NVIDIA H100

from$3.14/hr
Available now · 6 regions
Memory
80 GB HBM3
Bandwidth
up to 3.4 TB/s
Full pricing details

Ada Lovelace

NVIDIA RTX 4080

Not in stock right now
Memory
16 GB GDDR6X
Bandwidth
736 GB/s
Full pricing details

At a glance

Spec by spec. The longer bar is the better number.

Memory

NVIDIA H10080 GB
NVIDIA RTX 408016 GB

Memory bandwidth

NVIDIA H100up to 3.4 TB/s
NVIDIA RTX 4080736 GB/s

Full specs

The complete spec sheet.

SpecNVIDIA H100NVIDIA RTX 4080
ArchitectureHopperAda Lovelace
Memory80 GB16 GB
Memory typeHBM3GDDR6X
Memory bandwidthup to 3.4 TB/s736 GB/s
FP32up to 67 TFLOPS48.7 TFLOPS
FP16up to 989 TFLOPS, tensor97.5 TFLOPS, non-tensor
INT8up to 1979 TOPS390 TOPS
TDPup to 700 W320 W
GPUs per node82
Hourly price (live)from $3.14/hrNot in stock right now

Questions

H100 or RTX 4080. What buyers ask first.

Other comparisons

More pairs with the H100 or the RTX 4080.