> ## Documentation Index
> Fetch the complete documentation index at: https://runcrate.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Receipts and invoices to JSON to a webhook

> Photos of receipts to checked JSON with a vision model and a strict schema, escalated to a bigger model when the numbers do not add up, posted to your webhook.

Expense tools, bookkeeping and procurement all start with a photo of a receipt. A vision model with a strict JSON schema reads it, but the check that matters is arithmetic: do the lines add up to the subtotal, and the subtotal plus tax to the total? This pipeline runs the cheap model first, only escalates to the large one when the numbers disagree, and posts each result to your endpoint signed, keyed by the image's hash so a retry never books it twice. Three receipts cost six hundredths of a cent.

```mermaid theme={"theme":{"light":"github-light","dark":"catppuccin-mocha"}}
flowchart LR
  A[receipt.jpg] --> B[Extract JSON<br/>Qwen3-VL 30B]
  B --> C{lines add up?}
  C -- no --> D[Extract again<br/>Qwen3-VL 235B]
  C -- yes --> E[signed POST<br/>Idempotency-Key = sha256]
  D --> E
```

## Build it in your project

Paste this line into your agent (Claude Code, Codex, Cursor, OpenCode, Cline). It sets up your key, then builds the pipeline in your project:

```text theme={"theme":{"light":"github-light","dark":"catppuccin-mocha"}}
Add Runcrate's Qwen3 VL 30B A3B to this project: run curl -fsSL "https://runcrate.ai/agents.md?model=Qwen%2FQwen3-VL-30B-A3B-Instruct&intent=build" and follow it. Then build: a script that extracts receipts and invoices in a folder to strict JSON (vendor, date, currency, lines, subtotal, tax, total) with Qwen3-VL 30B, checks that the lines add up, retries once on Qwen3-VL 235B when they do not, and posts each result to WEBHOOK_URL with an HMAC signature and an idempotency key.
```

## The code

One file per language. Both read `RUNCRATE_API_KEY` from the environment; nothing secret is in the code.

```python receipts_to_webhook.py theme={"theme":{"light":"github-light","dark":"catppuccin-mocha"}}
"""Receipts and invoices to checked JSON, posted to your webhook.

pip install openai requests
RUNCRATE_API_KEY=... WEBHOOK_URL=https://your.app/hooks/receipts WEBHOOK_SECRET=... python receipts_to_webhook.py ./receipts
"""
import base64, hashlib, hmac, json, mimetypes, os, sys, time
from pathlib import Path

import requests
from openai import OpenAI

client = OpenAI(base_url="https://api.runcrate.ai/v1", api_key=os.environ["RUNCRATE_API_KEY"], max_retries=4)
FAST = "Qwen/Qwen3-VL-30B-A3B-Instruct"     # first pass
STRONG = "Qwen/Qwen3-VL-235B-A22B-Instruct"  # only when the numbers don't add up

SCHEMA = {
    "type": "object",
    "additionalProperties": False,
    "required": ["vendor", "date", "currency", "lines", "subtotal", "tax", "total"],
    "properties": {
        "vendor": {"type": "string"},
        "date": {"type": "string", "description": "ISO 8601, YYYY-MM-DD"},
        "currency": {"type": "string", "description": "ISO 4217 code"},
        "lines": {
            "type": "array",
            "items": {
                "type": "object",
                "additionalProperties": False,
                "required": ["description", "quantity", "amount"],
                "properties": {"description": {"type": "string"}, "quantity": {"type": "number"}, "amount": {"type": "number"}},
            },
        },
        "subtotal": {"type": "number"},
        "tax": {"type": "number"},
        "total": {"type": "number"},
    },
}

def extract(path: Path, model: str) -> tuple[dict, float]:
    mime = mimetypes.guess_type(path.name)[0] or "image/jpeg"
    data_url = f"data:{mime};base64,{base64.b64encode(path.read_bytes()).decode()}"
    raw = client.chat.completions.with_raw_response.create(
        model=model,
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": "Extract this receipt. Amounts are line totals as printed. Use the currency the receipt shows."},
                {"type": "image_url", "image_url": {"url": data_url}},
            ],
        }],
        response_format={"type": "json_schema", "json_schema": {"name": "receipt", "strict": True, "schema": SCHEMA}},
        temperature=0,
    )
    reply = raw.parse()
    return json.loads(reply.choices[0].message.content), float(raw.headers.get("x-llm-cost", 0))

def adds_up(r: dict) -> bool:
    lines = round(sum(l["amount"] for l in r["lines"]), 2)
    return abs(lines - r["subtotal"]) < 0.02 and abs(r["subtotal"] + r["tax"] - r["total"]) < 0.02

def post_webhook(payload: dict) -> int:
    url = os.environ.get("WEBHOOK_URL")
    if not url:
        return 0
    body = json.dumps(payload, separators=(",", ":")).encode()
    sig = hmac.new(os.environ.get("WEBHOOK_SECRET", "").encode(), body, hashlib.sha256).hexdigest()
    headers = {"Content-Type": "application/json", "X-Signature-SHA256": sig, "Idempotency-Key": payload["sha256"]}
    for attempt in range(5):
        try:
            res = requests.post(url, data=body, headers=headers, timeout=10)
            if res.status_code < 500 and res.status_code != 429:
                return res.status_code
        except requests.RequestException:
            pass
        time.sleep(2 ** attempt)
    raise RuntimeError(f"webhook {url} kept failing")

def main(folder: str) -> None:
    ledger = []
    for path in sorted(p for p in Path(folder).iterdir() if p.suffix.lower() in {".jpg", ".jpeg", ".png", ".webp"}):
        t0 = time.time()
        receipt, cost = extract(path, FAST)
        model = FAST
        if not adds_up(receipt):  # escalate once, to the bigger model
            receipt, extra = extract(path, STRONG)
            cost += extra
            model = STRONG
        payload = {
            "file": path.name,
            "sha256": hashlib.sha256(path.read_bytes()).hexdigest(),
            "model": model,
            "needs_review": not adds_up(receipt),
            "receipt": receipt,
        }
        status = post_webhook(payload)
        secs = time.time() - t0
        ledger.append({"file": path.name, "model": model, "seconds": round(secs, 2), "cost_usd": cost, "webhook": status, "total": receipt["total"]})
        print(f"{path.name}: {receipt['vendor']} {receipt['total']} {receipt['currency']}  {model}  {secs:.1f}s  ${cost:.6f}  webhook {status}")
    Path("run.json").write_text(json.dumps(ledger, indent=2))
    print(f"total ${sum(r['cost_usd'] for r in ledger):.6f} for {len(ledger)} receipts")

if __name__ == "__main__":
    main(sys.argv[1] if len(sys.argv) > 1 else "receipts")
```

```typescript receipts-to-webhook.ts theme={"theme":{"light":"github-light","dark":"catppuccin-mocha"}}
// Receipts and invoices to checked JSON, posted to your webhook.
//
// npm install openai
// RUNCRATE_API_KEY=... WEBHOOK_URL=https://your.app/hooks/receipts WEBHOOK_SECRET=... npx tsx receipts-to-webhook.ts ./receipts
import { createHash, createHmac } from "node:crypto";
import { readdirSync, readFileSync, writeFileSync } from "node:fs";
import { extname, join } from "node:path";
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.runcrate.ai/v1", apiKey: process.env.RUNCRATE_API_KEY, maxRetries: 4 });
const FAST = "Qwen/Qwen3-VL-30B-A3B-Instruct"; // first pass
const STRONG = "Qwen/Qwen3-VL-235B-A22B-Instruct"; // only when the numbers don't add up

type Receipt = {
  vendor: string;
  date: string;
  currency: string;
  lines: { description: string; quantity: number; amount: number }[];
  subtotal: number;
  tax: number;
  total: number;
};

const SCHEMA = {
  type: "object",
  additionalProperties: false,
  required: ["vendor", "date", "currency", "lines", "subtotal", "tax", "total"],
  properties: {
    vendor: { type: "string" },
    date: { type: "string", description: "ISO 8601, YYYY-MM-DD" },
    currency: { type: "string", description: "ISO 4217 code" },
    lines: {
      type: "array",
      items: {
        type: "object",
        additionalProperties: false,
        required: ["description", "quantity", "amount"],
        properties: { description: { type: "string" }, quantity: { type: "number" }, amount: { type: "number" } },
      },
    },
    subtotal: { type: "number" },
    tax: { type: "number" },
    total: { type: "number" },
  },
};

const MIME: Record<string, string> = { ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".png": "image/png", ".webp": "image/webp" };

async function extract(path: string, model: string): Promise<[Receipt, number]> {
  const dataUrl = `data:${MIME[extname(path).toLowerCase()]};base64,${readFileSync(path).toString("base64")}`;
  const { data, response } = await client.chat.completions
    .create({
      model,
      messages: [
        {
          role: "user",
          content: [
            { type: "text", text: "Extract this receipt. Amounts are line totals as printed. Use the currency the receipt shows." },
            { type: "image_url", image_url: { url: dataUrl } },
          ],
        },
      ],
      response_format: { type: "json_schema", json_schema: { name: "receipt", strict: true, schema: SCHEMA } },
      temperature: 0,
    })
    .withResponse();
  return [JSON.parse(data.choices[0].message.content ?? "{}"), Number(response.headers.get("x-llm-cost") ?? 0)];
}

function addsUp(r: Receipt): boolean {
  const lines = Math.round(r.lines.reduce((s, l) => s + l.amount, 0) * 100) / 100;
  return Math.abs(lines - r.subtotal) < 0.02 && Math.abs(r.subtotal + r.tax - r.total) < 0.02;
}

async function postWebhook(payload: { sha256: string }): Promise<number> {
  const url = process.env.WEBHOOK_URL;
  if (!url) return 0;
  const body = JSON.stringify(payload);
  const sig = createHmac("sha256", process.env.WEBHOOK_SECRET ?? "").update(body).digest("hex");
  for (let attempt = 0; attempt < 5; attempt++) {
    try {
      const res = await fetch(url, {
        method: "POST",
        body,
        headers: { "Content-Type": "application/json", "X-Signature-SHA256": sig, "Idempotency-Key": payload.sha256 },
        signal: AbortSignal.timeout(10_000),
      });
      if (res.status < 500 && res.status !== 429) return res.status;
    } catch {
      // network error: retry
    }
    await new Promise((r) => setTimeout(r, 2 ** attempt * 1000));
  }
  throw new Error(`webhook ${url} kept failing`);
}

const folder = process.argv[2] ?? "receipts";
const ledger: { file: string; model: string; seconds: number; cost_usd: number; webhook: number; total: number }[] = [];
for (const name of readdirSync(folder).filter((f) => MIME[extname(f).toLowerCase()]).sort()) {
  const path = join(folder, name);
  const t0 = Date.now();
  let [receipt, cost] = await extract(path, FAST);
  let model = FAST;
  if (!addsUp(receipt)) {
    // escalate once, to the bigger model
    const [again, extra] = await extract(path, STRONG);
    receipt = again;
    cost += extra;
    model = STRONG;
  }
  const sha256 = createHash("sha256").update(readFileSync(path)).digest("hex");
  const status = await postWebhook({ file: name, sha256, model, needs_review: !addsUp(receipt), receipt } as { sha256: string });
  const seconds = (Date.now() - t0) / 1000;
  ledger.push({ file: name, model, seconds, cost_usd: cost, webhook: status, total: receipt.total });
  console.log(`${name}: ${receipt.vendor} ${receipt.total} ${receipt.currency}  ${model}  ${seconds.toFixed(1)}s  $${cost.toFixed(6)}  webhook ${status}`);
}
writeFileSync("run.json", JSON.stringify(ledger, null, 2));
console.log(`total $${ledger.reduce((s, r) => s + r.cost_usd, 0).toFixed(6)} for ${ledger.length} receipts`);
```

## What the proof run cost

The Python file on three receipt photos made for the test (rotated and slightly blurred), with known totals. Measured on 2026-10-23 against `api.runcrate.ai`; costs are what the account was billed, read from each response's `X-LLM-Cost` header or `usage.cost`.

| Step | Model or tool | Time | Cost |
| - | - | - | - |
| Blue Door Coffee, GBP 10.05 | Qwen/Qwen3-VL-30B-A3B-Instruct | 5.9 s | \$0.000211 |
| Hardware Hub, USD 48.97 | Qwen/Qwen3-VL-30B-A3B-Instruct | 5.2 s | \$0.000213 |
| Taqueria La Luna, USD 24.24 | Qwen/Qwen3-VL-30B-A3B-Instruct | 5.2 s | \$0.000205 |
| Total, 3 receipts, 3 webhooks (signature verified) | | 16 s | \$0.000629 |

All three totals, vendors, dates and currencies were exact, in both languages (TypeScript: \$0.000629, 17 s). Every receipt added up, so the escalation to the 235B model did not run.

## Scale it

* Thousands a day: each receipt is independent; run a thread pool, or a [job](https://runcrate.ai/docs/jobs/overview.md) group over a folder on a volume.
* PDF invoices: render each page to an image first (`pdftoppm -r 150`), then send the pages in one message.
* Cap the spend with a [key limit](https://runcrate.ai/docs/models/authentication.md); a broken loop stops at your number.

## Gotchas found in the proof run

* `inclusionAI/Ling-3.0-flash-VL` is cheaper but ignored `response_format` and answered in prose; Qwen3-VL honoured the strict schema every time.
* Ask for line totals "as printed": otherwise the model sometimes multiplies quantity by unit price itself and disagrees with the receipt.
* Sign the exact bytes you send (`separators=(",", ":")` in Python) or the receiver's HMAC will not match.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.