OpenClaw’s Model-Call Telemetry PR Makes Prompt Cost Debuggable at the Span Where It Happens

Every agent cost investigation eventually starts with the same miserable question: which part of the prompt got expensive? The invoice says tokens. The trace says a model call happened. The runtime says the agent was “working.” None of that tells you whether the bill came from duplicated system instructions, a ballooning tool schema, stale context, or one plugin that quietly turned every turn into a novella.

OpenClaw PR #95770 is a useful correction because it puts prompt-shape metadata on the model-call span where the cost is actually incurred. The patch adds safe prompt metadata and provider usage attributes to diagnostic events and OpenTelemetry spans: current-turn input message count and characters, system prompt characters, tool definition count and characters, total prompt characters, plus per-call usage when the provider exposes it. Raw prompts and outputs remain behind the existing diagnostics.otel.captureContent policy.

That last sentence is the line between observability and a data breach wearing a dashboard.

“Just log the prompt” is not an observability strategy

There is a lazy version of LLM observability that solves every debugging problem by storing the full prompt and completion. It is seductive because it works during development. It is also exactly how teams end up shipping private Slack threads, customer records, internal code, secrets, and tool outputs into whichever observability backend had the friendliest trial signup.

PR #95770 takes the more mature path. The new metadata namespace, openclaw.model_call.prompt.*, tracks prompt composition without exposing prompt content by default. Operators can see input message volume, system prompt size, tool definition burden, and total prompt characters. Usage attributes land under openclaw.model_call.usage.* and gen_ai.usage.* when available from the provider. Existing aggregate accounting remains on openclaw.model.usage.

That split matters. Aggregate run accounting is useful for budgets and summaries, but it is too coarse for debugging. If a delegated task costs twice as much after a plugin install, the operator needs to know whether each call got heavier, whether there were more calls, whether a failover moved work to a pricier provider, or whether tool definitions started dominating the prompt. Run-level totals answer “how bad was it?” Per-call spans answer “where did it happen?”

OpenClaw’s OpenTelemetry docs already position the official diagnostics-otel plugin as an OTLP/HTTP protobuf exporter for diagnostics covering token usage, cost, run duration, failover, tool execution, oversized payloads, exec, and memory pressure. They also note that provider calls can receive a W3C traceparent header from OpenClaw’s trusted model-call span context when the transport accepts custom headers. PR #95770 makes that trace more useful by attaching the prompt budget to the call, not just to the eventual accounting rollup.

Prompt shape is now an operations metric

This is bigger than OpenClaw housekeeping. The agent market is learning that cost control is not solved by picking a cheaper model. Coding agents are loops. They assemble context, expose tools, call models, run commands, observe results, compact history, retry failures, and sometimes fork background work. Token spend is an emergent property of that loop.

That is why per-call prompt composition should be treated like latency, error rate, and memory pressure. A sudden increase in tool definition characters is not trivia. It may mean a plugin registered too many tools globally, a descriptor cache failed, or a channel integration exposed unnecessary actions to every turn. A growing system prompt can indicate policy duplication or stale migration text. Input-message bloat may point to session-history repair pulling too much context forward. None of those problems are visible if the only metric is total tokens per conversation.

Practitioners should build dashboards around this shape, not merely around spend. Track system prompt chars, tool definition chars, input message chars, model-call duration, provider name, model name, usage tokens, failover count, and run outcome. Alert on step changes after upgrades. Compare prompt composition before and after enabling new plugins. Watch for tool definitions consuming a large share of the prompt on turns that do not need tools. If your “cheap” model requires twice as many retries because prompt state is messy, it may not be cheap.

This also connects to the broader usage-based billing shift around coding agents. GitHub Copilot credits, Codex token budgets, Qwen-style stats, OpenCode limits, Langfuse pricing audits, and internal FinOps reviews all point in the same direction: agent cost is becoming an engineering concern, not a procurement footnote. Finance can set budgets. Engineering has to make the loop observable enough to stay inside them.

Privacy-preserving telemetry is the grown-up default

The right default for agent telemetry is metadata first, content by explicit policy. That is especially true for developer tools, where prompts routinely include source code, stack traces, environment details, credentials accidentally pasted into chat, and private project context. OpenClaw’s existing diagnostics.otel.captureContent gate is the right place for raw content. Most teams should be able to answer “why did this call get large?” without recording the exact user text or tool output.

There is still hard work ahead. Character counts are not exact token counts across providers, and different model tokenizers can turn the same prompt into different bills. Provider usage fields are uneven. Tool schemas can matter not only by length but by complexity and placement. But “imperfect prompt-shape telemetry on the span” is a large improvement over “look at the invoice and guess.”

The verification notes are appropriately boring: tests for model diagnostic events and the diagnostics OTel service, a docs format check for docs/gateway/opentelemetry.md, git diff --check, and local autoreview with no accepted actionable findings. This is the kind of patch where boring is the point. Observability should be uneventful until the day it saves you from a runaway prompt.

The editorial take: agent cost control is not a pricing-page problem. It is an observability problem. OpenClaw is putting the prompt budget where operators can inspect it — on the model-call span — while keeping raw content behind a policy gate. That is the shape of a production-grade answer.

Sources: OpenClaw PR #95770, OpenClaw OpenTelemetry docs, OpenTelemetry GenAI semantic conventions