NVIDIA H100 vs NVIDIA A40: which is better for AI?

Choose NVIDIA H100 if you need 80GB+ on a single GPU (32GB more than NVIDIA A40). Choose NVIDIA H100 if you need FP16 tensor throughput (561% faster than NVIDIA A40).

What's the price difference between NVIDIA H100 and NVIDIA A40?

On Runcrate today, NVIDIA H100 starts at $4.29/hr and NVIDIA A40 starts at $2.42/hr; NVIDIA A40 is cheaper by 44%.

Can I deploy NVIDIA H100 or NVIDIA A40 on Runcrate today?

NVIDIA H100 is on demand in 4 regions from $4.29/hr, billed by the second on most GPUs. NVIDIA A40 is on demand in 6 regions from $2.42/hr, billed by the second on most GPUs.

How much VRAM does each GPU have?

NVIDIA H100 has 80GB HBM3. NVIDIA A40 has 48GB GDDR6.

GPU comparison

NVIDIA H100 vs NVIDIA A40.

Specs, performance and live cloud availability, side by side. The NVIDIA A40 is cheaper today, from $2.42/hr.

Launch A40 Get a reserved quote

The short answer

Choose NVIDIA H100 if you need 80GB+ on a single GPU (32GB more than NVIDIA A40).
Choose NVIDIA H100 if you need FP16 tensor throughput (561% faster than NVIDIA A40).

Compare another pair

Hopper

NVIDIA H100

from$4.29/hr

Available now · 4 regions

Memory: 80 GB HBM3
FP16 tensor: up to 989 TFLOPS
Bandwidth: up to 3.4 TB/s

Launch H100 Get a reserved quote

Full pricing details

Ampere

NVIDIA A40

from$2.42/hrCheaper

Available now · 6 regions

Memory: 48 GB GDDR6
FP16 tensor: 149.7 TFLOPS
Bandwidth: 696 GB/s

Launch A40 Get a reserved quote

Full pricing details

At a glance

Spec by spec. The longer bar is the better number.

Memory

NVIDIA H10080 GB

NVIDIA A4048 GB

Memory bandwidth

NVIDIA H100up to 3.4 TB/s

NVIDIA A40696 GB/s

FP16 tensor

NVIDIA H100up to 989 TFLOPS

NVIDIA A40149.7 TFLOPS

Hourly price, from (live)

NVIDIA H100$4.29/hr

NVIDIA A40$2.42/hr

Full specs

The complete spec sheet.

Spec	NVIDIA H100	NVIDIA A40
Architecture	Hopper	Ampere
Memory	80 GB	48 GB
Memory type	HBM3	GDDR6
Memory bandwidth	up to 3.4 TB/s	696 GB/s
FP32	up to 67 TFLOPS	37.4 TFLOPS
FP16 tensor	up to 989 TFLOPS	149.7 TFLOPS
INT8	up to 2400 TOPS	299 TOPS
TDP	up to 700 W	300 W
GPUs per node	8	4
Hourly price (live)	from $4.29/hr	from $2.42/hr

Questions

H100 or A40. What buyers ask first.