Skip to main content
Compare GPU performance before committing to a configuration. Deploy instances with different GPUs, run standardized benchmarks, and choose the right GPU for your workload.

1. Deploy benchmark instances

2. Wait for the benchmark tools

The Inference template installs PyTorch and vLLM a few minutes after boot. This returns once they are in:

3. FP16 matrix multiplication benchmark

Test raw compute throughput: run on each instance to compare:

4. Memory bandwidth test

5. Check GPU specs

Expected results

Tips

  • Run benchmarks 3 times and average: GPU boost clocks vary between runs.
  • FP16 matmul tests compute-bound workloads (training). Memory bandwidth tests memory-bound workloads (inference).
  • The RTX 4090 offers the best price-to-performance for inference. The H100 is best for training throughput.

Cleanup