Metrics and the real cost of context

Every metric should drive a decision#

Most metric catalogues list what can be measured. This one adds the column they omit: the decision each metric drives. A metric that drives no decision is decoration, and should be deleted.

If you track only five#

Together they cover the five constraints and the one blind spot. Everything else is diagnosis once one of these moves.

The full catalogue#

The four-line cost model#

Agent cost has four lines. Most teams instrument one.

Line 1: compute. Input tokens are 99.75–99.87% of agent token usage S. Output is a rounding error. Any cost model that reports "tokens" without splitting input by cache state can be off by an order of magnitude. This chapter uses a generic price shape, in relative units per 1K tokens:

The 12.5× spread between a cached read and a new write is why cache behaviour dominates.

Line 2: the cache-write premium. New tokens cost more than plain uncached input, because writing them into the cache carries a premium. That is why churn is expensive: a token that enters, is invalidated and re-enters is paid at 1.25 twice, instead of 0.10 per turn if it had simply stayed.

Line 3: latency. Compaction is a blocking call that can stall the agent for tens of seconds S. Deferred tool loading adds a round trip. Sub-agents add orchestration overhead. Latency becomes money through engineers waiting.

Line 4: people. Instruction-file upkeep, memory curation, tool audits, shaping scripts, transcript review. A 30-minute quarterly audit is cheap. A memory system that needs weekly curation is not.

Two beliefs the arithmetic overturns#

Belief 1: "Cutting tokens cuts cost" C#

False whenever the cut is in the prefix. A team replaces a static 38K tool surface with per-turn dynamic selection, cutting definitions to 9K — chapter 3's dynamic-tool trap C.

A 24% token cut produced a 6.9× cost increase, because the tool block sits about 10K into the prompt and changing it invalidates everything after it C.

Belief 2: "Compaction saves money" C#

True, but far less than the token count suggests, and only if the session continues.

Take a 120K context compacted to 40K (a 25K stable prefix that stays cached, plus a 15K new summary), with 2.5K added per turn afterwards. The compaction call itself — reading the 120K transcript at the cached rate (12.0) and writing the 15K summary at 1.25 (18.75) — costs about 30.8 units.

Afterwards, each turn costs about 7.1 units (25K prefix and 15K summary re-read cached, 2.5K new written) rising 0.25 per turn as earlier additions re-enter cached — the compacted series above is that arithmetic, cumulative from the 30.8-unit call.

Two consequences.

1. Compacting near the end of a session is a pure loss. You pay the full price and collect none of the saving. With fewer than about five turns left, do not compact.
2. The saving is modest because the kept context was already cheap. Holding 80K extra cached tokens costs 8 units per turn (80 × 0.10), not the 100 units the raw count implies. Meanwhile the quality cost is undiminished: the 6.5-point SWE-bench summariser swing, the Pass² drop and the 44.6% termination rate are paid in full S.

Worked monthly economics C#

A five-engineer team, each running about six agent sessions a day, 20 working days a month.

Sessions per month          = 5 × 6 × 20                 = 600
Turns per session                                        = 45
Mid-session context                                      = 110K
Prefix tax (3 MCP servers, 380-line instruction file)    = 62K
New tokens per turn                                      = 2.6K

Cost per turn    = (110 − 2.6) × 0.10 + 2.6 × 1.25       = 13.99 units
Cost per session = 45 × 13.99                            = 629.55 units
Cost per month   = 600 × 629.55                          = 377,730 units

Prefix tax            62K → 6K    (tool audit + inference test)
Mid-session context  110K → 48K   (shaping + offload + just-in-time)
New tokens per turn   2.6K → 1.9K (output shaping)

Cost per turn    = (48 − 1.9) × 0.10 + 1.9 × 1.25        = 6.99 units
Cost per session = 45 × 6.99                             = 314.55 units
Cost per month   = 600 × 314.55                          = 188,730 units

Compute saving: 50%. Now the lines everyone forgets.

Where the value actually is. The cost saving is real, but it is not the headline. Mid-session context fell from 110K to 48K, from 55% of a 200K window to 24%. That gives the workload a lower-utilisation condition to compare with the higher one; any quality effect should be measured with chapter 9's design.

Cost per solved task#

The one number that captures cost and quality together:

cost_per_solved = total_cost / tasks_solved

It cannot be gamed in either easy direction. Cutting context aggressively lowers cost and lowers the solve rate; the ratio catches it. Dumping in more context raises solve rate slightly and cost sharply; the ratio catches that too.

This is the arithmetic form of a measured result: full context cost 2.68× the best method while completing fewer tasks S. It adds a lesson: you can over-compress as easily as you can over-load.

Where to spend the next unit of effort#

Run this checklist in order. Stop at the first "yes".

1. Cache hit rate under 60%? Fix the prefix. Everything else is downstream and may be self-defeating.
2. More than 3 tools defined per tool used? Inspect deletion candidates. If a representative sample confirms dead tools, remove them with rollback; the largest win is workload-specific.
3. Tool results over 35% of the session? Output shaping: one afternoon, 60–90% reduction P.
4. Read-utilisation under 5%? Structural retrieval.
5. Post-boundary re-fetch over 2? Offload before compaction; fix the schema.
6. Pass² ÷ Pass@2 under 0.85? You are shipping intermittency. Back off compression.
7. None of the above? Stop optimising context. Your binding constraint is elsewhere: model choice, task decomposition or the verification loop.

Step 7 is the one people skip. Context management has a stopping point, and continuing past it is how teams end up maintaining machinery that costs people's time and buys nothing.