Skip to main content
Process thousands of prompts through an LLM in a single session. Deploy a GPU, run vLLM in offline batch mode, collect results, tear down. Pay only for the compute hours you use.

1. Prepare your input file

Create a JSONL file with one prompt per line:

2. Deploy and upload

3. Upload the batch script

vLLM is installed on the Inference template. It finishes installing a few minutes after boot; runcrate ssh batch-job -- "systemctl start runcrate-python" returns once it has.

4. Download results and tear down

Monitor progress

Time estimate

Multiply the time by the GPU’s rate on the live GPU pricing page to estimate cost.

Tips

  • vLLM offline batch mode uses continuous batching: much faster than sequential API calls.
  • For 100K+ prompts, split the file and process in chunks to avoid OOM.
  • Check your balance before starting: runcrate billing balance.