Your Token Bill: A Mystery Novel in Four Acts

This is satire, but it is also a reconstruction of the crime. At 9:07 a.m., the engineering team of Parchment & Sons, a fictional SaaS company specializing in enterprise comma placement, receives an AI invoice large enough to qualify as a minor moon. Nobody knows why. The agent was only asked to “fix the flaky test.” It produced a six-line patch, an apology, and 19 million ceremonial tokens devoted to rereading the repository’s historical relationship with YAML.

Act I: The Body Is Found in the Context Window

The first suspect is output. This is natural: output is visible, verbose, and occasionally ends with a 43-line explanation of why a renamed variable reflects the team’s values. But output is usually the corpse’s least interesting accessory. The useful question is what the tool sent in before it began talking: system instructions, chat history, repository maps, retrieved files, tool definitions, test logs, screenshots, and the agent’s previous attempts to rediscover that the service runs on port 8080.

At Parchment & Sons, investigators run the traditional command:

rg -n "(readFile|glob|search|git diff|git log)" .agent/runs/latest.jsonl | wc -l

The result is 4,812. This is not evidence of intelligence. It is evidence that the agent has developed a rich inner life centered on opening the same lockfile in four different tools. In the real world, request usage can be broken down by model, request count, input tokens, output tokens, and cached input tokens when the provider exposes those dimensions. If your dashboard only says “usage increased,” you have not received observability. You have received a weather report from a curtain.

The fix is unglamorous: put a ceiling on the amount of repository material any task begins with. Start with the package manifest, the target files, the failing test, and the relevant configuration. Do not automatically attach the entire monorepo because the issue contains the word “auth.” An agent cannot distinguish “useful context” from “every file that has ever made a sound” unless the harness makes that distinction first.

Act II: The Cache Has an Alibi

The second suspect is caching, which arrives wearing a badge and saying it has already seen this prefix. It may be telling the truth. Provider pricing documentation distinguishes ordinary input, cache writes, cache reads or hits, and output; cache durations and rates can differ too. That means “we enabled caching” is not a postmortem. It is the beginning of one.

The forensic detail is prefix stability. Put the stable material first: system prompt, tool schemas, repository instructions, and the mostly immutable project map. Put the timestamp, current branch SHA, request ID, and the agent’s latest emotional weather at the end. If a different random identifier is injected at byte 37 on every request, the cache is not being defeated by advanced economics. It is being defeated by a string formatter.

The Department of Cache Investigations recommends this entirely reasonable policy: any prompt prefix longer than 8,000 tokens must be signed by the Vice President of Repeating Oneself. Less ceremoniously, log cache-hit tokens beside uncached input tokens. If the latter is high on every turn, inspect which early prompt segment changes. If the cached number abruptly drops after compaction or truncation, that is a behavior to understand, not a spiritual betrayal by the model.

Act III: The Agent Develops a Taste for Process

The third suspect is tool use. A coding agent that can run tests, inspect logs, search code, browse docs, call a second model, and ask a third model to critique the second has discovered the procurement department’s favorite concept: a workflow. Every tool result returns to context. A 2 MB test log is not a test result; it is a novella being invoiced repeatedly.

Parchment & Sons solves this with a script that is so radical it prints fewer things:

npm test -- --runInBand 2>&1 | tail -n 120

They also make shell tools return structured summaries where possible: exit code, changed files, failing test names, and the first relevant stack frames. Save the full log as an artifact. Fetch it only when the agent has a reason. This is bad for the agent’s freedom, which is precisely why it works. Agents are notably bad at deciding whether a 14,000-line build log contains one useful line when reading all 14,000 lines costs somebody else’s money.

Set a run budget before the agent starts: maximum turns, maximum tool calls, maximum wall-clock time, and an explicit stop condition such as “if two test runs fail without a smaller diff, stop and ask.” A budget of 20 turns is not a performance benchmark. It is a circuit breaker with a number printed on it.

Act IV: The Invoice Explains Itself, Almost

In the final act, the chief financial detective discovers that the team has been comparing unlike things. One tool charges by tokens. Another includes an allowance and measures premium interactions or credits. Another bills model calls but leaves vector storage, hosted tools, or external search to separate line items. The invoice is not mysterious because math is hard; it is mysterious because every product has invented a different unit called “simple.”

GitHub’s Copilot documentation is a useful period detail: GitHub moved to usage-based billing on June 1, 2026, where cost depends on model and token consumption, while certain existing annual subscribers could remain on a legacy premium-request system. That is not an indictment of GitHub. It is a reminder to record the plan, date, model, and billing scheme before comparing two months of spend. Otherwise the dashboard will confidently tell you that March was cheaper than April because March was measured in apples and April in executive raisins.

The final control is boring enough to survive contact with reality: export usage daily, group it by model and API key or project, and attach a run ID to agent jobs. Track input, cached input, output, tool calls, retries, and elapsed time. Then sample the ten most expensive runs and read the traces. You will find a few legitimate expensive jobs, a few avoidable loops, and one agent that spent forty-seven minutes proving a file did not exist.

The joke ends here because the true observation is too ordinary to improve: token cost is usually a product of what your system repeatedly sends, not what the model finally says. The bill becomes less mysterious the moment a human can connect each expensive run to a prompt, a tool trace, and a stop rule.