> ## Documentation Index > Fetch the complete documentation index at: https://runcrate.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Observability tools > Send every model call your environments make to Langfuse, Datadog, an OpenTelemetry backend, PostHog or your own webhook, within a minute. Every call to the inference API is on [Models > Observability](https://runcrate.ai/dashboard/models/observability) for 30 days. If your team already watches its model calls somewhere else, Runcrate sends each call there too: its model, tokens, cost, how long it took, its TTFT, its status, and the end user, session and tags you sent with it. ## How it works **1. Connect the tool** In [Settings > Integrations](https://runcrate.ai/dashboard/settings/integrations), select **Connect** beside the tool and paste its key. Runcrate sends one sample call first and saves the connection only if the tool accepts it. Keys are encrypted at rest and never shown again. Owners and managers can connect tools. **2. Pick the environments** On [Models > Observability](https://runcrate.ai/dashboard/models/observability), the **Send to** line names the connected tools. Select **Choose per environment** and tick the tools each environment sends to. An environment sends nowhere until you pick one, and a tool gets calls made from the moment it was picked, not the days before. **3. Watch the calls arrive** New calls are sent in batches every minute. The tool's row in Integrations shows the last batch it took, the last error in plain words and how many calls are still to send. **Test** sends one more sample call and shows the tool's answer. ### Prompts and answers A call is sent without its prompt and answer unless both of these are on: 1. the environment saves prompts and answers ([Settings > Environments](https://runcrate.ai/dashboard/settings/environments)), and 2. the tool's **Send prompts and answers** switch (off by default). Prompts and answers saved as cut short are sent as saved; images, audio and video are never sent. ### When a tool fails If a tool refuses a batch or does not answer within 15 seconds, Runcrate keeps the batch and tries again after 1, 2, 4, 8 and 16 minutes, then every 30 minutes, for 24 hours. Calls are always sent in order and none is skipped: a tool that is down for an hour gets that hour's calls once it is back. After 24 hours of failures the tool is **Paused**. Fix the cause (a new key, a new address) and select **Resume**: the calls it missed are sent while they are still kept (30 days). A batch can arrive twice (after an answer that came too late). Each call keeps the same ids every time it is sent: its trace and span ids, its PostHog event `uuid`, its `data[].id` on the webhook, so a repeat can be told apart. What is never sent: which supplier served a call, or what it cost us. ## ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/langfuse.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=6ca8c19e45759dd86c514f049fdb1732) Langfuse Calls go to Langfuse's OpenTelemetry endpoint (`/api/public/otel/v1/traces`, on its v4 data model, so they show at once). Each call is one **generation** (an **embedding** for `/v1/embeddings`) in a trace of its own: | Langfuse | From the call | | - | - | | Trace name | The endpoint and model, such as `chat meta-llama/Llama-3.3-70B-Instruct` | | `metadata.request_id` | The `X-Request-Id` the API returned | | `model` | The model id, such as `meta-llama/Llama-3.3-70B-Instruct` | | `usageDetails` | `input`, `output`, `input_cached_tokens`, `input_cache_write_tokens`, `output_reasoning_tokens`, `total` | | `costDetails.total` | What the call was billed, in USD | | `startTime`, `completionStartTime`, `endTime` | When it arrived, its first token, its last byte | | `userId`, `sessionId` | The end user and session you sent | | `tags` | Your tags, as `key:value` | | `environment` | The Runcrate environment's name, in lowercase | | `level`, `statusMessage` | `ERROR` and the error, for a failed call | | `input`, `output` | The prompt and answer, when both switches are on | **Feedback** you send for a call (thumbs or a rating, from the API, MCP, the playground or the dashboard) becomes a `NUMERIC` **score** on its generation, from -1 to 1, named by its label or `feedback`, with your comment. A changed rating replaces its score. To connect: pick the region (EU cloud, US cloud or your own host, Langfuse v3.22 or later) and paste a **public key** (`pk-lf-…`) and **secret key** (`sk-lf-…`) from your project's **Settings > API keys**. ## ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/datadog.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=88bdb68a95f0c93ac225cf68aff995bc) Datadog Choose how each call arrives: * **A log** (every Datadog plan): `source:runcrate`, your service name, `status:error` for a failed call, tags `env`, `model`, `endpoint`, `source`, `session_id` and your own, the end user as `usr.id`, and the call under the `runcrate` attribute (tokens, cost, timings, status). * **An LLM Observability span** (where your account has LLM Observability): one `llm` span per call under your **ML app** name, with `model_name`, input and output tokens, TTFT, cost and status, and the prompt and answer as messages when both switches are on. To connect: pick your **site** (the region in your Datadog address: US1, US3, US5, EU, AP1, AP2 or US1-FED) and paste an **API key** from **Organization settings > API keys**. An application key will not work. ## ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/opentelemetry.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=36f98e9b2245bd9a6dabd7405f7b1a94) OpenTelemetry Any backend that takes OTLP over HTTP. Each call is one `CLIENT` span, named like `chat meta-llama/Llama-3.3-70B-Instruct`, with the [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/): | Attribute | From the call | | - | - | | `gen_ai.operation.name` | `chat`, `text_completion`, `embeddings` or `generate_content` | | `gen_ai.provider.name` | `runcrate` | | `gen_ai.request.model`, `gen_ai.response.model` | The model id | | `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | Tokens in and out | | `gen_ai.usage.cache_read.input_tokens` | Cached input tokens | | `gen_ai.response.finish_reasons` | Why it stopped | | `gen_ai.conversation.id` | Your session | | `gen_ai.input.messages`, `gen_ai.output.messages` | The prompt and answer, when both switches are on | | `user.id` | Your end user | | `error.type`, span status | The error, for a failed call | | `runcrate.*` | Request id, environment, cost (USD), TTFT, and `runcrate.tag.` per tag | To connect: paste the OTLP/HTTP **endpoint** (Runcrate adds `/v1/traces` unless it is there) and the **headers** your backend wants. Start from one of these: | Backend | Endpoint | Header | | - | - | - | | ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/honeycomb.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=e8e38bdd4c76dde84ed03a0c8035505f) Honeycomb | `https://api.honeycomb.io` | `x-honeycomb-team: ` | | ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/grafana.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=0ddc5a8c22374b480bba8ba7d1ce04c7) Grafana Cloud | `https://otlp-gateway-prod-.grafana.net/otlp` | `Authorization: Basic ` | | ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/signoz.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=ba98990c03db4ffe2c80f64ed4ad9d04) SigNoz Cloud | `https://ingest..signoz.cloud:443` | `signoz-ingestion-key: ` | | ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/elastic.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=b4c391412c8f488271c5572882030d6f) Elastic | Your deployment's APM endpoint | `Authorization: ApiKey ` | The address must be public and use https. ## ![](https://mintcdn.com/runcrate/9rCVrp0IpPQiCf6V/images/integrations/posthog.svg?fit=max\&auto=format\&n=9rCVrp0IpPQiCf6V\&q=85\&s=4d73a318bc6a9912988bd08ec159201a) PostHog Each call is a `$ai_generation` event in PostHog's **LLM analytics**: `$ai_model`, `$ai_provider` (`runcrate`), `$ai_input_tokens`, `$ai_output_tokens`, `$ai_total_cost_usd`, `$ai_latency` and `$ai_time_to_first_token` in seconds, `$ai_http_status`, `$ai_is_error` and `$ai_error`, `$ai_trace_id` (the call's id), `$ai_session_id`, and `$ai_input` and `$ai_output_choices` when both switches are on. Your tags arrive as `tag_` properties. The event's person is your end user. A call without one is sent under `runcrate:` without a person profile, so anonymous traffic does not fill your persons list. **Feedback** is a `$ai_metric` event on the call's trace (and a `$ai_feedback` event when it has a comment). To connect: pick US cloud, EU cloud or your own host and paste the **project API key** (`phc_…`) from **Project settings**. It can only send events, never read them. ## Webhook Your own HTTPS endpoint gets each batch as one JSON `POST`, signed like every Runcrate webhook: ```http theme={"theme":{"light":"github-light","dark":"catppuccin-mocha"}} POST /your/endpoint Content-Type: application/json Runcrate-Event: inference.requests Runcrate-Delivery: 6f1d1a0e-8a5b-4d3c-9a43-0d6c9f0f2a11 Runcrate-Signature: t=1792756800,v1=5d2f8c0b9e...e91a ``` ```json theme={"theme":{"light":"github-light","dark":"catppuccin-mocha"}} { "type": "inference.requests", "api_version": "2026-10-23", "id": "6f1d1a0e-8a5b-4d3c-9a43-0d6c9f0f2a11", "created_at": "2026-10-23T12:05:00.000Z", "workspace_id": "1c8c7fa0-0497-4600-b284-16609d52d991", "data": [ { "id": "a0000000-0000-4000-8000-000000000001", "request_id": "req_8f2c", "created_at": "2026-10-23T12:00:00.000Z", "environment": { "id": "e0000000-0000-4000-8000-000000000001", "name": "Production" }, "api_key_id": "k0000000-0000-4000-8000-000000000001", "endpoint": "chat", "model": "meta-llama/Llama-3.3-70B-Instruct", "status": 200, "outcome": "ok", "error": null, "latency_ms": 1500, "ttft_ms": 200, "tokens": { "input": 100, "cached_input": 20, "cache_write": 0, "output": 50, "reasoning": 0 }, "cost_usd": 0.00012, "stream": true, "source": "api", "free": false, "end_user": "user-42", "session_id": "sess-7", "tags": { "team": "search" }, "finish_reason": "stop", "output_tokens_per_sec": 41.5, "tool_calls": 1 } ] } ``` With both switches on, each call also carries `prompt`, `answer` and `body_truncated`. Feedback arrives as its own batch, `"type": "inference.feedback"`, with `request_id`, `call_id` (the call's `id`), `score` (-1 to 1), `rating`, `label`, `comment` and `source`. `Runcrate-Signature` is the hex HMAC-SHA256 of `t`, a full stop and the raw body, keyed with the signing secret Runcrate shows once when you connect the webhook. Verify it with the code in [verifying signatures](https://runcrate.ai/docs/webhooks/verifying-signatures.md). Answer with a 2xx within 15 seconds; anything else is retried as above. ## Related > **[Verifying signatures](https://runcrate.ai/docs/webhooks/verifying-signatures.md)** > > Check the Runcrate-Signature header on your webhook. > **[Webhooks](https://runcrate.ai/docs/webhooks/overview.md)** > > Events about instances, jobs and billing, posted as they happen. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.