Skip to content

GPU comparison

H100 vs MI300X.

The MI300X is ahead on memory (192 GB vs 80 GB) and speed (at least 32% more FP16 throughput and at least 58% more memory bandwidth). The H100 makes sense when your model fits in 80 GB and its hourly price is lower.

Rent either on Runcrate

Live hourly price, then per GB of memory and per 100 FP16 TFLOPS.

Which is better for

Which to pick for each job. Memory fit and throughput, from the datasheets.

  • Hold a 70B model in 8-bit weights

    70 GB of 8-bit weights

    H100: 1 GPU, 10 GB spare

    MI300X: 1 GPU, 122 GB spare

    MI300X. Same GPU count, at least 58% more memory bandwidth.

  • Serve an 8B model in 16-bit weights

    16 GB of 16-bit weights

    H100: 1 GPU, 64 GB spare

    MI300X: 1 GPU, 176 GB spare

    MI300X. Same GPU count, at least 58% more memory bandwidth.

  • Fine-tune a 70B model with QLoRA

    About 46 GB with 4-bit weights

    H100: 1 GPU, 34 GB spare

    MI300X: 1 GPU, 146 GB spare

    MI300X. Same GPU count, at least 32% more FP16 throughput.

  • Train on a full node

    FP16 tensor throughput per node

    H100: 8x, up to 7,912 TFLOPS

    MI300X: 8x, 10,459 TFLOPS

    MI300X. at least 32% more FP16 throughput per GPU.

  • Throughput per watt

    FP16 TFLOPS per 100 W of TDP

    H100: Varies by card

    MI300X: 174 TFLOPS per 100 W

    Either. Varies by card.

Compare another pair

vs

Hopper

NVIDIA H100

from$3.15/hrCheaper
Available now · 3 regions
Memory
80 GB HBM3
FP16 tensor
up to 989 TFLOPS
Bandwidth
up to 3.4 TB/s
Full pricing details

CDNA 3

AMD MI300X

from$4.85/hr per GPU

on an 8-GPU machine

Available now · 1 region
Memory
192 GB HBM3
FP16 tensor
1307.4 TFLOPS
Bandwidth
5.3 TB/s
Full pricing details

At a glance

Spec by spec. The longer bar is the better number.

Memory

NVIDIA H10080 GB
AMD MI300X192 GB

Memory bandwidth

NVIDIA H100up to 3.4 TB/s
AMD MI300X5.3 TB/s

FP16 tensor

NVIDIA H100up to 989 TFLOPS
AMD MI300X1307.4 TFLOPS

Hourly price, from (live)

NVIDIA H100$3.15/hr
AMD MI300X$4.85/hr

Full specs

The complete spec sheet.

SpecNVIDIA H100AMD MI300X
ArchitectureHopperCDNA 3
Memory80 GB192 GB
Memory typeHBM3HBM3
Memory bandwidthup to 3.4 TB/s5.3 TB/s
FP32up to 67 TFLOPS163 TFLOPS
FP16 tensorup to 989 TFLOPS1307.4 TFLOPS
INT8up to 1979 TOPS2610 TOPS
TDPup to 700 W750 W
GPUs per node88
Hourly price (live)from $3.15/hrfrom $4.85/hr per GPU on an 8-GPU machine

Questions

H100 or MI300X. What buyers ask first.

Other comparisons

More pairs with the H100 or the MI300X.