Spend guardrails

​

Max runtime

• Ten minutes before the limit, owners and admins get an email and your webhooks get an instance.runtime_ending event with the exact stop time.
• At the limit the instance is terminated, not paused. Its local disk is deleted, as with a manual terminate. Keep anything you need on a storage volume or push it somewhere before the limit.
• The range is 5 minutes to 7 days (10,080 minutes). Leave it unset for no limit.
• Once the instance is running, auto_stop_at on the instance tells you when it will stop: its start time plus the max runtime.

Instances > Launch an instance > Auto stop > Stop after > 4 hours

runcrate instances create --name training-run --gpu H100 --max-runtime 4h

const instance = await rc.instances.create({
  name: 'training-run',
  sshKeyId: 'key_abc123',
  gpuType: 'H100',
  maxRuntimeMinutes: 240,
});

instance = client.instances.create(
    name="training-run",
    ssh_key_id="key_abc123",
    gpu_type="H100",
    max_runtime_minutes=240,
)

curl https://www.runcrate.ai/api/v1/instances \
  -H "Authorization: Bearer rc_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "training-run", "gpu_type": "h100-sxm", "max_runtime_minutes": 240}'

{
  "tool": "create_instance",
  "arguments": { "name": "training-run", "gpu": "H100", "max_runtime_minutes": 240 }
}

​

Idle stop

• The range is 5 minutes to 24 hours (1,440 minutes). Leave it unset to keep the instance running while idle.
• An open SSH, web terminal or code editor session counts as activity, so an instance you are working on is not stopped while you are connected.
• The idle clock starts once the instance has finished setting up.
• Like max runtime, idle stop terminates the instance and deletes its local disk.
• It applies to instances launched with the setting. Instances launched before idle stop existed are not watched.

runcrate instances create --name notebook --gpu A100 --idle-stop 30m

await rc.instances.create({ name: 'notebook', sshKeyId: 'key_abc123', gpuType: 'A100', idleShutdownMinutes: 30 });

client.instances.create(name="notebook", ssh_key_id="key_abc123", gpu_type="A100", idle_shutdown_minutes=30)

{ "name": "notebook", "gpu_type": "a100-sxm", "idle_shutdown_minutes": 30 }

​

Changing auto stop on a running instance

• Max runtime still counts from when the instance started running, so give the total minutes, not extra minutes. A change never stops the instance sooner than 2 minutes from the moment you make it, and auto_stop_at shows the new stop time.
• Extending the limit sends a fresh instance.runtime_ending alert 10 minutes before the new stop time.
• null (CLI off) turns a limit off. A field you leave out keeps its value.
• An idle stop change reaches the instance within about 5 minutes, and restarts its idle clock. Some older instances can’t turn idle stop on (409, “Idle stop needs a newer launch”), and neither can CPU-only instances; turning it off always works.
• Only a starting or running instance can change (409 otherwise).

Instances > the instance's menu > Change auto stop

runcrate instances auto-stop training-run --max-runtime 8h --idle-stop off

await rc.instances.update(instance.id, { maxRuntimeMinutes: 480, idleShutdownMinutes: null });

client.instances.update(instance.id, max_runtime_minutes=480, idle_shutdown_minutes=None)

curl -X PATCH https://www.runcrate.ai/api/v1/instances/training-run \
  -H "Authorization: Bearer rc_live_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"max_runtime_minutes": 480, "idle_shutdown_minutes": null}'

{
  "tool": "set_instance_auto_stop",
  "arguments": { "id": "training-run", "max_runtime_minutes": 480 }
}

​

Monthly budgets

• Who sets it: a workspace owner or admin, in Settings, Environments.
• What counts: GPU instances and inference billed to that environment, including inference made with that environment’s API keys.
• The month: the UTC calendar month. Month to date resets to $0 at 00:00 UTC on the 1st.
• At 80%: owners and admins get an email and your webhooks get a budget.threshold_reached event.
• At 100%:
  • New launches in the environment fail with HTTP 402 and error code BUDGET_REACHED.
  • Running instances in the environment are terminated, with stop_reason set to budget.
  • Inference calls with the environment’s API keys fail with HTTP 402: error type insufficient_quota, code budget_exceeded.
  • Owners and admins get an email and your webhooks get a budget.reached event.

​

Reading a budget

{
  "data": {
    "id": "e81b4f6a-2c3d-4a9e-b710-5f8c2d1e9a04",
    "name": "agents",
    "is_default": false,
    "monthly_budget_usd": 500,
    "month_to_date_usd": 412.37,
    "budget_remaining_usd": 87.63
  }
}

{
  "error": {
    "code": "BUDGET_REACHED",
    "message": "This environment has reached its monthly budget."
  }
}

​

Alerts

• Email to every active owner and admin of the workspace.
• Webhooks: your own HTTPS endpoint, a Slack channel or a Discord channel, added by an owner or admin in Settings, Webhooks. See webhooks to set one up.

​

Stop reasons

​

Related