What you’ll build
A fine-tuned model adapter that specializes an open-source LLM for your use case: customer support, medical QA, code generation, legal analysis, or anything else. The adapter merges back into the base model and can be served with vLLM.GPU sizing
LoRA rank selection
Step-by-step (CLI)
1. Prepare your dataset
Create a JSONL file with your training data:2. Deploy a GPU and upload your data
3. Dependencies
The Finetune template (ubuntu-train) has PyTorch for CUDA 12.8, Transformers, Datasets, PEFT, TRL, Accelerate and bitsandbytes installed. They finish installing a few minutes after boot; this returns once they are in:
pip install keeps that PyTorch.
4. Training script (finetune.py)
5. Run training
finetune.py and the data on a volume first (paths under /workspace instead of /root), then:
6. Monitor training
7. Download the adapter and clean up
8. Merge and serve
After downloading, merge the adapter into the base model locally or on another instance:Using the Python SDK
Using MCP (via Claude Code / Cursor)
“Spin up an A100 called ‘finetune’. Install torch, transformers, peft, trl, accelerate, and bitsandbytes. Then show me a training script for QLoRA fine-tuning Llama 3.1 8B.”The agent deploys the instance, installs packages via
ssh_execute, and generates the training script for you.