Skip to content

GPU comparison

H200 vs A100 80GB.

The H200 is ahead on memory (141 GB vs 80 GB) and speed (3.2x the FP16 throughput and at least 2.3x the memory bandwidth). The A100 80GB makes sense when your model fits in 80 GB and its hourly price is lower.

Rent either on Runcrate

Live hourly price, then per GB of memory and per 100 FP16 TFLOPS.

Which is better for

Which to pick for each job. Memory fit and throughput, from the datasheets.

  • Hold a 70B model in 8-bit weights

    70 GB of 8-bit weights

    H200: 1 GPU, 71 GB spare

    A100 80GB: 1 GPU, 10 GB spare

    H200. Same GPU count, at least 2.3x the memory bandwidth.

  • Serve an 8B model in 16-bit weights

    16 GB of 16-bit weights

    H200: 1 GPU, 125 GB spare

    A100 80GB: 1 GPU, 64 GB spare

    H200. Same GPU count, at least 2.3x the memory bandwidth.

  • Fine-tune a 70B model with QLoRA

    About 46 GB with 4-bit weights

    H200: 1 GPU, 95 GB spare

    A100 80GB: 1 GPU, 34 GB spare

    H200. Same GPU count, 3.2x the FP16 throughput.

  • Train on a full node

    FP16 tensor throughput per node

    H200: 8x, 7,912 TFLOPS, NVLink

    A100 80GB: 8x, 2,496 TFLOPS

    H200. 3.2x the FP16 throughput per GPU.

  • Throughput per watt

    FP16 TFLOPS per 100 W of TDP

    H200: 141 TFLOPS per 100 W

    A100 80GB: Varies by card

    Either. Varies by card.

Compare another pair

vs

Hopper

NVIDIA H200

from$4.28/hr
Available now · 5 regions
Memory
141 GB HBM3e
FP16 tensor
989 TFLOPS
Bandwidth
4.8 TB/s
Full pricing details

Ampere

NVIDIA A100 80GB

from$1.73/hrCheaper
Available now · 6 regions
Memory
80 GB HBM2e
FP16 tensor
312 TFLOPS
Bandwidth
up to 2.0 TB/s
Full pricing details

At a glance

Spec by spec. The longer bar is the better number.

Memory

NVIDIA H200141 GB
NVIDIA A100 80GB80 GB

Memory bandwidth

NVIDIA H2004.8 TB/s
NVIDIA A100 80GBup to 2.0 TB/s

FP16 tensor

NVIDIA H200989 TFLOPS
NVIDIA A100 80GB312 TFLOPS

Hourly price, from (live)

NVIDIA H200$4.28/hr
NVIDIA A100 80GB$1.73/hr

Full specs

The complete spec sheet.

SpecNVIDIA H200NVIDIA A100 80GB
ArchitectureHopperAmpere
Memory141 GB80 GB
Memory typeHBM3eHBM2e
Memory bandwidth4.8 TB/sup to 2.0 TB/s
FP3267 TFLOPS19.5 TFLOPS
FP16 tensor989 TFLOPS312 TFLOPS
INT81979 TOPS624 TOPS
TDP700 Wup to 400 W
GPUs per node88
Hourly price (live)from $4.28/hrfrom $1.73/hr

Questions

H200 or A100 80GB. What buyers ask first.

Other comparisons

More pairs with the H200 or the A100 80GB.