Skip to main content
Before deploying, ask your AI agent to compare GPU options. It can pull live pricing, filter by VRAM requirements, and recommend the best option for your workload.

​
”Compare A100, H100, and RTX 4090 pricing. I need at least 40GB VRAM.”

The agent queries available GPUs:
  1. list_gpu_types — fetches all available GPU types with pricing and specs
GPUs with 40GB+ VRAM: For your use case:
  • If you’re fine-tuning a 7B model, the A6000 ($0.60/hr) has enough VRAM and saves 75% vs H100.
  • If you’re serving a 70B model, you need the A100 80GB minimum. The H100 gives ~40% more throughput for 56% more cost.
  • The RTX 4090 (24 GB) doesn’t meet your 40GB requirement.

​
”What’s the cheapest GPU that can run Llama 3.1 70B?”

The agent calculates VRAM requirements and matches:
  1. list_gpu_types — gets all GPU options
Llama 3.1 70B needs ~70 GB VRAM at BF16, or ~35 GB at 8-bit quantization. Cheapest single-GPU options: Multi-GPU option (lower per-GPU cost): Recommendation: Single A100 80GB at 1.60/hr.It′sthecheapestpathto70Bservingwithfullprecision.The2xA10040GBsetupcostsmore(1.60/hr. It's the cheapest path to 70B serving with full precision. The 2x A100 40GB setup costs more (2.40/hr) and adds tensor-parallel overhead.

​
”Show me everything available under $1/hr for quick experiments.”

The agent filters by price:
  1. list_gpu_types — gets all GPU types, filters by hourly rate
GPUs under $1/hr: Best value for quick experiments: RTX 4090 at $0.35/hr. It has the best price-to-performance ratio for single-GPU workloads — fast enough for real training, cheap enough to leave running during iteration. A 4-hour experiment on RTX 4090 costs $1.40 total.

​
Tools used in this workflow