Part I · Understand the problemChapter 1

Why more context can hurt coding agents

The question What is context, mechanically, and when does adding more make coding agents less reliable?

10 min read Foundational Step 1 of 20 1/2 in this part
Research contents
  1. Understand the problem
    1. 1Why more context can hurt
    2. 2Anatomy of the context
  2. The methods
    1. 3The ten methods
    2. 4Choosing your methods
    3. 5Retrieval
    4. 6Compaction, offload, memory
    5. 7Sub-agents and isolation
    6. 8The tool surface
  3. Measure it
    1. 9Evaluating a change
    2. 10Metrics and economics
  4. Decide and avoid
    1. 11Hard calls
    2. 12Failure modes
    3. 13Antipatterns
    4. 14Case studies
  5. Act on it
    1. 15The optimisation plan
    2. 16The diagnostic
    3. 17Habits and tips
    4. 18Tooling landscape
  6. Reference
    1. ATemplates
    2. BLessons

Reference

On this page

In 30 seconds

  • Across Chroma's controlled tasks, performance generally fell as input grew, but the curves were model- and task-dependent S.
  • Five constraints shape a context decision: dilution, position, recency bias in conflicts, cache and information loss.
  • Context fails in five ways. Starvation can leave no trace in the transcript, so it needs a read-coverage check.

You will be able to

  • Explain the productive-band model without treating it as a universal curve
  • Route an observed symptom to one of five context constraints
  • Diagnose poisoning, distraction, confusion, clash and starvation from their tells
  • State the four properties of a well-managed context

Start with the right mental model#

Most people picture the context window as a container. It has a capacity. You put things in. When it is full, something has to come out. A bigger container means fewer problems.

That picture is wrong, and it produces most of the bad advice in this field. The contents of a container do not interfere with each other. Adding tokens changes the model's input and can change which evidence it uses.

A better picture is an auction for attention. The currency is not space. It is salience: how much useful signal the model can extract from the current input. Use this as an operating model, not a literal claim that every model exposes one fixed attention pool; the effect depends on model, task and position D. Three consequences follow.

  1. Adding correct information can lower performance. Distractors and competing interpretations can make the relevant evidence harder to use.
  2. More capacity is not more reliable use. The useful set grows until the next token adds more retrieval work or ambiguity than signal.
  3. "Does it fit?" is necessary, not sufficient. Measure whether the added material changes the task outcome.

A conceptual model: context value has a productive band

A conceptual three-zone model: too little context starves the task, a small relevant set is productive, and extra distractors dilute it. STARVED decisive evidence has not arrived PRODUCTIVE small, relevant set makes the next action inferable DILUTED distractors and competing interpretations more material in context →
Figure 1. Working model, not a quantitative curve: the productive band depends on the model, task and position. Long-context studies show that irrelevant or competing context can reduce retrieval reliability S; applying that result to coding agents is an inference D.

What long-context studies actually show#

The claim that models get worse as input grows is not folklore. It has been measured carefully. (The small letters after each number are evidence labels; S means a sourced study.)

Chroma's context-rot study tested 18 models from four vendors/model families on controlled long-context tasks. It covered 8 input lengths and 11 needle positions S. Across the experiments, performance generally degraded as input grew, with non-uniform, model- and task-specific curves S.

The third column below is an engineering inference for repository agents, not a direct coding-agent measurement D.

Study result S What was measured Engineering implication D
Similarity matters When the question and the answer used similar words, models found the answer even in long contexts. When they did not, performance fell much faster S. You ask "why is checkout slow?" and the answer is a connection-pool setting. Vocabulary mismatch is one source of difficulty; repository size is not the only determinant.
Distractors compound One near-miss distractor hurt; four hurt more; and which distractor mattered S. Codebases are full of near-duplicates: three parse_config functions, a deprecated AuthService next to the live one.
Coherent text is harder Shuffled haystacks produced better retrieval than coherent ones, across all 18 models S. A coherent design document may create a similar retrieval risk; test that on your workload. Coherence can create plausible wrong answers.
Focused beats full On LongMemEval, ~300 tokens of relevant text beat the full ~113K-token context retaining the same answer-bearing material plus surrounding context S. The focused condition retained the answer-bearing material while removing surrounding context. That is a useful subtraction test for your own agent.

The practical lesson is to retrieve by relevance and verify against the repository, not to assume that a coherent document is automatically useful.

Position is a task-dependent risk#

The cited Lost in the Middle study found a U-shaped position effect on its retrieval tasks: relevant information near the beginning or end performed better than information in the middle S. The magnitude varies by model and task, so do not turn it into a universal 50% utilisation threshold.

Position can change retrieval reliability (schematic)

In the cited long-context retrieval tasks, relevant information near the start or end performed better than information in the middle. The effect varies by model and task. START favoured in the cited retrieval tasks MIDDLE lower retrieval in the cited tasks END favoured in the cited retrieval tasks same fact · different placement · different reliability
Figure 2. The cited Lost in the Middle tasks found higher retrieval at the beginning and end than in the middle. This is schematic; the effect depends on model, task and harness S.

The practical response is placement plus verification: keep a critical constraint near the action it governs, then test whether late-session behavior actually improves. System-prompt placement is harness-dependent.

Advertised length is not effective length#

Advertised length is therefore not a promise of task reliability. Suites differ in construction, so their scores are not directly comparable; the detailed benchmark record belongs on the Sources page. See effective context length.

Five constraints around a context decision#

These are the research's operating categories, not five literal forces in every model. Use the first signal you can observe to choose what to measure and change.

Route the symptom to a first move

A diagnostic map from an observed context symptom to one of five constraints and a first engineering move. OBSERVED SIGNAL CONSTRAINT FIRST MOVE Irrelevant material dominates Dilution Narrow retrieval and remove dead context A rule is ignored late Position Restate it near the action; test again Two facts conflict Recency bias in conflicts Reset with handoff; save state first Cost jumps after a prompt edit Cache Stabilise the prefix; price the rewrite An important detail disappears Information loss Offload first; compress second
Figure 3. A first-pass routing map. The categories overlap in real failures; choose the most observable signal, then verify the change D.
Constraint What changes Measure
Dilution More candidates compete for relevance. Track relevance density, not window utilisation. Relevant tokens ÷ total tokens; compare a narrow read with a complete one.
Position Retrieval depends on where material lands and on the task. Place the same fact at different positions in a small repeat test.
Recency bias in conflicts A later claim can dominate an earlier one when facts conflict; this is model- and harness-dependent D. Find whether the first appearance of a claim came from tool output or an assistant message.
Cache Prefix caching stops at the first changed byte P. Cache hit rate and cache-adjusted cost; see chapter 10.
Information loss Summaries, masks and truncation remove detail; only some cuts are reversible. Re-fetch rate after compaction; record what was offloaded and addressable.

For information loss, the decisive distinction is recoverability:

Loss type Recoverable? Example
Reversible offload Yes: the content still exists and has an address Tool output written to a file, replaced in context by its path
Lossy compression No: a paraphrase cannot be inverted An LLM summary of the transcript
Hard truncation No, unless it was logged elsewhere Dropping the oldest turns first
Masking Depends: masked-but-kept is recoverable, masked-and-dropped is not Replacing old tool output with placeholders

Reversibility is usually worth more than compression ratio. The measured addressable-recall comparison is in chapter 6 S.

A reset clears poisoned history, but it also discards unrecorded state; write a handoff before paying that cost.

Five ways context fails#

Drew Breunig describes four failure modes. This research adds starvation because a missing file can leave no transcript trace.

The five failure modes, and how visible each one is

Poisoning, distraction, confusion and clash are usually inspectable in the transcript. Starvation is often silent. Poisoning a false fact becomes canon Distraction the agent copies its own history Confusion irrelevant context gets used Clash contradictions get blended Starvation a needed fact never arrives USUALLY INSPECTABLE OFTEN SILENT survives review
Figure 4. Four failures are usually inspectable in the transcript. Starvation can leave a clean-looking trajectory and a wrong answer, so it is easy to miss in review.

The diagram gives the taxonomy. This table gives the diagnostic action.

Failure The tell Containment
Poisoning A confident symbol or API claim first appeared in an assistant message, not tool output. Compaction may launder it into a settled fact. Restart from before the false claim and reground it in tool output.
Distraction The same tool and arguments recur without new information. Reset with a short note of what was tried and ruled out; instrument a loop detector.
Confusion The agent chooses something applicable but wrong for the situation, such as web search instead of local grep. Remove irrelevant material or tools. The measured tool-count effect is in chapter 8.
Clash Output blends current and withdrawn requirements. Keep one current decision record; treat the transcript as evidence, not state.
Starvation The decisive file was never read, so the trajectory looks clean and ends wrong. Run a read-coverage audit and improve retrieval.

Constraints describe what pushes context off course. Failure modes describe the resulting symptom. Use the constraint map to choose what to measure, then this table to choose containment. The operations below describe where to intervene.

Why coding-agent context is distinctive#

The mechanisms above are general. Coding agents make them especially consequential because context contains mutable repository state, executable evidence and large tool outputs.

Property What it means Practical consequence
The repository is the working surface and evidence source The agent edits what it reads. A previously read file can become stale after the agent changes it. Re-read before you re-edit; use just-in-time retrieval.
Verification is executable Tests, compilers and linters turn many claims into machine-checkable results. Let the test suite determine how much compaction risk you can afford.
Tool output is large and often disposable A pytest run or install log may contain thousands of tokens where one line matters. Shape output first; detailed measurements belong in chapter 2 and chapter 18.
Structure is a precise retrieval signal Directory layout, import graphs, symbol tables and test-to-source mapping can be recomputed from source. Start with structural retrieval; add semantic retrieval where vocabulary gaps justify it.

The operations you can perform on context#

A useful operational taxonomy is four transformations plus a prevention gate: write, select, compress and isolate, with prevent stopping waste before it enters. This adapts LangChain's write/select/compress/isolate frame and adds prevention as a preceding gate P.

A common operating order, not a requirement

A common, non-mandatory operating order: prevent waste, write recoverable state, select what is needed, compress after offloading, and isolate when a new boundary is useful. PREVENT stop waste before it enters WRITE offload and leave an address SELECT retrieve only what is needed COMPRESS reduce after offloading ISOLATE new scope, then select compressing can return to selection; isolation starts a new selection scope
Figure 5. Prevention reduces what enters context; the remaining operations decide what survives, enters, shrinks or is isolated. This is a common order, not a requirement; verify the loop for your harness D.
Operation Coding-agent examples Main risk
Prevent Output shaping, ignore files, fewer tools, quiet flags Hiding a line you needed
Write Scratch files, plan files, decision logs, memory Staleness or memory nobody reads
Select grep, file reads, symbol lookup, recall The wrong thing, or too much
Compress Compaction, summaries, masking Losing the decisive fact for good
Isolate Sub-agents, session resets, per-task scope Losing shared understanding

The loop matters: offload before compressing; after isolation, select again in the new scope. Compression rewrites the prompt and can fight the cache P.

What "good" looks like#

Here is a definition that later chapters can be checked against.

Each clause is a testable property.

Property Violated by How to test it
Sufficiency: the next action is inferable Starvation Read-coverage audits on failures
Minimality: the smallest such set Distraction, confusion, dilution Relevance density; remove a segment and see if the solve rate moves
Recoverability: what is missing can be fetched Lossy compaction with no offload After each compaction, ask: could the agent get that back?
Economy: it costs less than it saves Elaborate memory nobody queries The cost model in chapter 10

This programme therefore optimises for sufficiency, minimality, recoverability and economy—not completeness for its own sake.

Your next action#

Take one failed coding-agent run. Mark the decisive context as starvation (missing), distraction or confusion (irrelevant), clash (conflicting), poisoning (false/unverified), or information loss (unrecoverable). Record the first observable tell, then choose one response: retrieve, remove, offload or reset. Use the context postmortem, then carry the diagnosis into Chapter 2 or the retrieval chapter.

Key takeaways

  1. The window is an operating budget for useful signal. Adding correct information can still lower performance.
  2. Position can change retrieval reliability, but the size and shape depend on model, task and harness.
  3. Appending usually preserves the existing prefix; early edits can invalidate cached suffixes and may cost more than they save.
  4. The programme's operating preference is reversibility over compression ratio: prefer operations you can undo.
  5. You cannot delete from a transcript by talking to it. Reset instead of arguing.

Go deeper

Terms used in this chapter

  • Attention budget — The framing that replaces "context window as container": adding tokens can redistribute useful signal rather than creating more reliable attention.
  • Evidence label — A tag on a claim saying how it is supported: S (sourced), P (practitioner-reported), D (derived) or C (composite).
  • Context rot — The everyday name for performance becoming less reliable as input length grows. Chroma measured the pattern across 18 models, with non-uniform curves that depend on model and task [S].
  • Position decay — A critical fact becoming harder to use because its position in a long input is less favourable.
  • Effective context length — The length at which a model still performs acceptably on a given task, as opposed to the length it advertises; it may be shorter than the advertised maximum.
  • Relevance density — Tokens that could plausibly be cited in a correct answer, divided by all tokens in context. The working measure of dilution.
  • Prefix caching — Reusing the computed state of an unchanged prompt prefix. Prefix-exact: valid up to the first changed byte.
  • Laundering — Compaction turning a hedged hypothesis into an asserted fact by stripping the uncertainty around it.
  • Loop detector — A check that flags the same tool called with the same arguments three or more times with no new information in between.
  • Confusion — A failure mode where irrelevant context is used because it is present. The common case is picking a plausible wrong tool when too many are in scope.
  • Read-coverage — Whether the decisive file was ever opened during a failed task. It separates starvation from dilution, which need opposite fixes.