Compaction, offload and memory

Three things people confuse#

Three different mechanisms all get called "memory". Mixing them up leads to bad architecture.

What the 2026 research actually established#

Several independent research efforts converged in 2026. Together they should change how you configure compaction today.

Result 1: The summariser alone is worth several points#

Holding the agent completely fixed and changing only the summarising model, SWE-bench accuracy moved from 49.0% to 55.5%, a 6.5-point swing S.

This is the most decision-relevant number in the chapter. Compaction is not plumbing that either works or does not. It is a component with a quality dimension comparable to a model upgrade. A team that has never read its compaction prompt is leaving several points on the table.

Training on compacted trajectories adds more: +5.5 and +7.0 points on SWE-bench Verified (two model sizes) and +6.8 and +3.1 on Terminal-Bench 2.0 S. Most teams cannot train, but the direction is clear: handling compaction is a skill that can be improved.

Result 2: Compression damages reliability before accuracy#

This finding changes practice the most, and almost nobody has absorbed it.

The TRACE study compared compression strategies on AppWorld, a 147-task benchmark of stateful API use. It measured both single-run accuracy and Pass²: solved on both of two independent runs S.

The paper's own conclusion is the practical takeaway: compression quality is better reflected by multi-run stability than by single-run performance S.

Result 3: Compaction breaks "where am I?", and the next step is the worst#

Three measured effects S:

• Correct termination fell to 44.6% with summary replacement, against 77.2% for FIFO truncation, at a 2K budget. The agent loses the ability to recognise that it is done.
• +0.108 extra blocked or error actions at the first step after compaction.
• Regressive exploration: agents re-fetch and replay to recover what the summary dropped.

The mechanism: summaries preserve what happened much better than where we are. A fluent narrative leaves the agent unsure whether the current sub-goal is open or closed.

Two design consequences follow.

1. The summary must carry explicit state, not narrative: a done, in-progress, not-started list, not "we have been working on X".
2. Re-read the plan file immediately after compaction. The most dangerous step in the session is the one you can protect most cheaply.

Result 4: Lossless addressable compaction beats every lossy baseline#

Addressable recall compaction is deterministic, uses no model call, and is content-addressed. Old observations become citation stubs (an ID plus a head-and-tail preview), and the agent calls recall <id> when it needs one. It was compared against five baselines: full context, sliding window, LLM summary, structured state and RAG memory S.

It also saved 38.8–80.3% of memory bandwidth against a sliding window S. The paper states it bluntly: omitted or paraphrased details cannot be recovered from a summary.

Read the two rows honestly. The margin is huge on retrieval and modest on hard reasoning. Addressability fixes access to information; it does not make the model reason better with it.

Result 5: Simple masking is competitive with summarisation#

An independent comparison found that observation masking achieves solve rates comparable to LLM summarisation on SWE-bench across two model sizes, at substantially lower cost, because it needs no extra model call S. Its recommendation: try simple methods before building summarisation infrastructure.

Combined with Result 4, the order is clear, and it is the opposite of what most teams build:

Result 6: Semantic triggering beats both naive triggers#

Both simple triggers fail, in opposite directions — threshold too late, periodic mid-task S; the design space below picks the policy.

The self-compaction work gates on closed reasoning units: fire when a sub-task resolves or the trajectory converges; hold off mid-derivation or when stuck. That preserves verified facts that fixed-interval compaction destroys S. See semantic triggering.

The same literature adds two operational notes S. Summarisation is a blocking call that can stall the agent for tens of seconds. Prompt instructions about summary length are largely ignored. What the summary keeps also varies substantially from run to run — model calls are stochastic, and the variation is a direct cause of Result 2 D.

The compaction design space#

Six decisions. The recommendations follow from the results above.

Decision 1: the trigger#

Recommendation: semantic triggering as the policy, with a threshold backstop set high (85–90%) so it only fires when the policy failed. If you cannot change your harness's automatic trigger, compact manually at boundaries so the automatic one never fires.

Decision 2: what gets compressed#

Decision 3: the summary schema#

The highest-leverage prompt in your system (Result 1). "Summarise the conversation so far" leaves that on the table. Each section below maps to a measured failure.

## Goal
<one sentence: the original task, verbatim where possible>

## State                          ← counters the 44.6% termination collapse
- Done: <verified complete, and how it was verified>
- In progress: <the one current sub-goal>
- Not started: <what remains>

## Decisions
- <decision> — because <reason>

## Ruled out                      ← counters regressive exploration
- <approach> — because <reason>, evidence: <file:line or command>

## Exact strings (verbatim, do not paraphrase)   ← counters detail loss
- errors, versions, paths, line numbers, config keys, IDs

## Open questions

## Offloaded artifacts            ← counters irrecoverability
- <id or path> — <one-line description>

The full version, with operating rules, is in the templates appendix.

Decision 4: target size#

Smaller summaries lose more. But FIFO, which loses the most, kept state recognition better than summarisation. So the amount lost is not the only axis; what kind of loss matters more.

Recommendation: no fixed token target. Compress narrative aggressively; keep the state block and exact strings in full. A 1,200-token summary with verbatim error strings beats a 400-token one that paraphrased them.

Decision 5: how often#

Research setups cap at three compactions per run S. The practitioner rule is stricter: more than one compaction means the session should have ended.

Recommendation: cap at one, then reset with a handoff note. A second compaction compresses something already compressed. Errors compound, details are twice removed, and the variance effect of Result 2 stacks.

Decision 6: the cache#

Compaction rewrites everything after the stable prefix and invalidates the cache from that point. The worked arithmetic shows that compacting a 120K context to 40K breaks even after about four more turns C.

Two consequences. First, compacting near the end of a session is a pure loss: you pay and never collect. Second, because cached tokens were already cheap, compaction's money saving is much smaller than its token count suggests, while its quality cost is not. You pay measured reliability damage for a modest, slow saving.

Recommendation: compact less often and more decisively.

Offload: the pattern to reach for first#

Offload beats compaction wherever it applies, because it is reversible.

tool returns 14,000 tokens
  → write to  .agent/artifacts/a3f9.log
  → context receives:
      .agent/artifacts/a3f9.log (14.2KB)
      npm ci — exit 0, 2 peer-dep warnings (react-dom, @types/node)
  → the agent can `recall a3f9` or grep inside the store at any time

Three design rules, each learned from a failure:

1. The stub must carry enough to decide whether to recall it — a bare a3f9.log (14.2KB) is dead weight, as chapter 3's M-6 shows. Adding the exit code and a warning summary costs 15 tokens and turns a deleted file into a working pointer.
2. Do not offload what is still in use. Offloading the log from the command that just ran forces an immediate recall. Offload after a few turns or at a sub-goal boundary.
3. Keep the manifest bounded. Past about 30 items, an always-visible manifest becomes the problem it solved. Switch to a searchable store with a one-line index.

Where to put it. A gitignored scratch directory outside the source tree (for example .agent/) for logs and intermediate output. In the repository, versioned, for anything a person should review, such as plans and decision records. At this scale the filesystem beats a database: the agent already has file tools, there is no schema, and grep works.

Cross-session memory#

This is the area with the weakest evidence and the most enthusiastic tooling. It is treated sceptically for that reason.

What memory is for#

Only one use survives scrutiny: facts that were expensive to learn and stay true. Everything else belongs in the repository.

The test: would this change a decision, and would re-deriving it cost a lot? It needs both.

Three ways memory fails#

1. Write-only memory. The system collects thousands of entries and never surfaces a decisive one. Every memory tool demos writing; almost none measure reading. If you cannot report "memories retrieved that changed an action, per week", you do not know whether your memory works.
2. Staleness. A memory about code that has since been refactored is worse than nothing: confidently wrong and silent. Prefer memories about things that change slowly: infrastructure, conventions, contracts between systems, operational knowledge.
3. Prefix cost. Memory loaded at session start is prefix. A growing memory file is a growing tax on every call. Ten memories is a tool; a thousand is the tool-bloat problem again, with worse selection.

An architecture that survives the objections#

Tier 3: the repository does the most work#

The repository is the authoritative memory: versioned, reviewed, and repaired by the same process that changes the code. An ADR committed to the repository is a better memory than a database entry. It is discoverable by grep, updated alongside the code, and needs no MCP server.

The honest position on memory tooling: the ecosystem is large, the mechanisms are reasonable, and published evidence that it improves coding-agent outcomes is thin. Adopt it with a measurement plan, not on the strength of a demo.

External knowledge stores: wiki, database, graph#

Your organisation already has a wiki, probably a database or two, and possibly someone building a knowledge graph. What should the agent do with them?

The short answer is dereference, never index. The reasoning matters more than the rule, because it generalises to stores that do not exist yet.

The governing variable is staleness rate#

Every store can hold a fact. What separates them is what happens to that fact when the code changes underneath it. See staleness rate.

Only the first column has a repair mechanism. That is the whole argument. It is not that Markdown beats SQL. A fact living next to the code it describes is the only fact with a maintainer.

The wiki#

Most organisations' largest knowledge store, and the worst calibrated for agents.

Worse for an agent than for a person. Nothing fails when a wiki page rots, so its error rate only rises — and a person discounts a page by its last-modified date and has been burned before; an agent does neither. Worse still, a stale wiki page about exactly your subsystem is a maximally plausible distractor. Distractor damage rises with similarity to the query S, and a page describing the exact module you are working on, in your vocabulary, is at the top of that curve. Bulk-indexing the wiki is therefore not neutral. It manufactures high-similarity distractors D.

The migration, the one thing worth doing. A bounded, one-time extraction. Then stop reading the wiki.

1. List the pages the team actually opened last quarter. Ignore the long tail.
2. For each, ask: does a decision depend on this, and is it expensive to re-derive?
3. For each survivor, ask the harder question: where in the repository would this fail loudly if it became wrong? Next to the config it describes, in an ADR beside the decision, as a comment above the workaround, or best of all as a test.
4. Move it there, in a reviewed pull request.
5. Replace the wiki page with a link to its new home. This stops the two from drifting apart again.

Steps 3 and 5 are the ones teams skip, and skipping either turns a clean-up into a duplication.

What legitimately stays in the wiki: facts with no code home, such as on-call rotations, escalation paths, vendor contacts and team ownership. These are also, notably, facts a coding agent rarely needs.

When a database beats the filesystem#

The question is not whether to use a database. It is what makes the filesystem stop being enough.

Markdown files under ripgrep are versioned, diffable, reviewable, searchable with the same tools for people and agents, portable, and free to run. Below roughly ten thousand documents, nothing beats that D. A real store earns its place when one of these holds:

If you cross one, the order is SQLite with full-text search first, then Postgres, then anything else. SQLite keeps the property that mattered: it is still a file.

Knowledge graphs, and a disambiguation#

"Graph" means two completely different things in this research. Mixing them up is how graph-memory services get justified with structural retrieval's evidence.

Why knowledge graphs underperform for coding, strongest reason first:

1. The graph you want already exists, and is exact (see the table above).
2. Extraction is a poisoning surface. A false edge looks like schema, not opinion, and poisoning cannot be removed by appending a correction.
3. Agent questions do not match graph traversal. "Where is X defined?" and "what calls Y?" are grep and language-server questions. "Why is Z like this?" is not a graph query at all; it is an ADR.
4. Ingestion lag is unbounded. The code changes every merge; the graph changes when the pipeline runs.

The real exception: relationships no single compiler can see, such as service dependencies across repositories, data lineage across systems, ownership and on-call mapping, and API contracts between repositories. If your proposed graph covers one repository's source, the compiler already won.

Methods before tools#

Tools for this layer, with verdicts, are in chapter 18.

The complete policy#

Everything above as a decision record you could implement from:

prevention:
  output_shaping: true              # do this first
  ignore_files: [node_modules, dist, "*.lock", __generated__]

offload:                            # before any compression
  threshold_tokens: 1500
  delay_turns: 2
  stub_includes: [path, size, exit_code, one_line_summary]
  store: .agent/artifacts/
  manifest_max_items: 30

compaction:
  trigger:
    semantic: [subtask_complete, hypothesis_resolved, tests_pass]
    suppress: [mid_edit, stuck, immediately_after_error]
    backstop_threshold: 0.88
  strategy:
    observations: citation_stub     # reversible
    reasoning: llm_summary          # into the fixed schema
    plan: never                     # lives in PLAN.md
    exact_strings: verbatim
  max_per_session: 1
  post_compaction: reread PLAN.md   # protects the highest-error step

memory:
  tier1_max_lines: 40
  tier2: searchable_markdown
  tier3: the_repository             # ADRs, tests, comments
  metric: retrievals_that_changed_an_action

external_stores:
  wiki: {bulk_index: false, autonomous_search: false, fetch_by_name: true, treat_as: hypothesis}
  database: {default: sqlite_fts5, vector_only: false}
  knowledge_graph: {over_source_code: false, cross_system_only: true}
  staleness_audit: {cadence: quarterly, sample: 20, delete_above_false_rate: 0.20}

session:
  reset_after: [second_compaction_needed, loop_detected, poison_found]

Measuring the compression layer#

If you add only one, add Pass² ÷ Pass@2. It measures Result 2 directly, no single-run evaluation can see it, and it predicts the complaint you are about to get.