Retrieval: how code should enter context

The question, posed correctly#

"Semantic search versus grep" is a category error. It compares a technology with a tool and ignores the variables that actually decide the outcome. Retrieval design has four independent axes.

"Grep versus embeddings" collapses all four into one. A pre-loaded, file-level vector search and a just-in-time, symbol-level grep differ on every axis. Crediting the result to "embeddings" credits the wrong variable.

The consequence is uncomfortable: a retrieval comparison run on someone else's harness does not transfer to yours — the study above included. It tells you the shape of the answer (lexical search is a strong baseline; do not skip it). It does not tell you the size of the effect on your setup. Chapter 9 shows how to measure it yourself.

Why code is an unusual corpus#

Most retrieval intuition comes from document search. Code breaks five of its assumptions, and each break changes what works.

The four paradigms#

Lexical: grep and ripgrep#

How it works. Pattern matching over file contents. No index, no staleness, no build step.

Strengths. Exact, complete within its query, fast on any repository size. It finds what lives outside the symbol graph: string literals, config keys, comments, YAML, SQL, environment variables, feature flags.

Weaknesses. You have to know or guess the token. It returns line fragments, which causes fragment blindness: a matched line shown with three lines of context, cut off before the if that governs it, read as if it always runs. The fix is trivial: use -C 8 or expand to the enclosing function.

Evidence. Generally more accurate than vector retrieval in the four-harness study S. Eight surveyed coding agents use the model as a navigator over shell tools such as grep, find and ripgrep S. This is the majority architecture, not a fallback.

Structural: LSP, tree-sitter, symbol index#

How it works. Parse the code. Expose symbols, definitions, references and the code graph. Retrieve symbol bodies and traverse exact edges.

Strengths. The retrieval unit matches the meaning unit: a 40-line function instead of a 1,200-line file. Reference traversal has no false positives. It answers questions grep cannot express: what calls this, what implements this interface, what breaks if this signature changes. Mature toolkits cover 30+ languages via the Language Server Protocol P.

Weaknesses. Needs a working language server, with setup, memory and warm-up. It has a silent failure mode: a timeout returns "no results", which looks exactly like "does not exist". It cannot see configuration, string-keyed dispatch, schemas, CI definitions or templates.

Evidence. Mostly mechanical rather than benchmarked: the waste argument and the exact-edge argument are both large and derivable. The composite rename example in chapter 3 shows about 15× fewer tokens with better precision C.

Semantic: embeddings and vector search#

How it works. Chunk the repository, embed the chunks, embed the query, return nearest neighbours.

Strengths. Answers questions when you do not know the vocabulary: "how do we handle rate limiting?" when the code says throttle. Useful over non-code material such as design documents, ADRs and PR discussions.

Weaknesses. Index staleness, worst on recently edited code. Chunk boundaries that split functions. Approximate answers where exact ones exist. And the one that matters most: it always returns something. Grep returning zero hits is information. A vector search returning five irrelevant chunks is misinformation that looks like success.

Evidence. Generally less accurate than grep head-to-head S. A hybrid of semantic search and grep has been reported 12.5% more accurate than either alone P. Production systems routinely combine semantic, keyword (BM25) and metadata filters P.

Live agentic search: the LLM as navigator#

How it works. Not a fourth index but a control strategy. The agent iterates: search, read, decide the next query, stop when satisfied. See agentic search.

Strengths. The query adapts to what was found. It handles the common case where the first search is wrong because the first hypothesis was wrong.

Weaknesses. Turns and latency. Quality depends on the model's search skill, which is exactly where the harness confound bites hardest. Practitioners report agents spending 60%+ of their time searching for context P.

Evidence. No benchmark isolates the control strategy's contribution; the four-harness spread is the closest signal D.

The comparison#

The recommended architecture#

Synthesised from the evidence, as a default to override with your own measurements.

Two rules govern the tiers.

1. Escalate, do not parallelise. Try tier 1. If the identifier is unknown or the results are empty, go up a tier. Running every tier on every query multiplies tokens and floods context with three overlapping result sets.
2. Expand along the graph, not across the file. Having found a hit, read its callers, implementers and tests, not the lines above and below it. Spatial adjacency in code is a weak signal; graph adjacency is a strong one. This one habit is worth more than most index choices.

Nine ways code retrieval goes wrong#

Two of these are counterintuitive. R-5 is the agent poisoning its own context with its own work: the more productive the session, the more of what it read is now wrong. R-7 means good documentation can hurt. Coherent text retrieves worse than shuffled text across all 18 models tested S. That does not mean stop writing design documents. It means do not pre-load them into a coding agent. Fetch them on demand for questions about intent, and treat the code as the authority on behaviour.

Should you seed the session with a codebase overview?#

The evidence against. The same three results chapter 3's M-3 cites: focused beats full on every measured comparison, and full context costs 2.68× for fewer completed tasks SP.

The evidence for a small seed. Just-in-time retrieval fails by starvation when the agent does not know a subsystem exists. Human-written context files improved success by about 4% P — small and positive, though the figure has no published method.

The synthesis: pre-load pointers and invariants, never content.

Keep it under 2,000 tokens. For any candidate line, ask: does this help the agent decide where to look, or does it try to replace looking? The first is worth its tokens many times over. The second is the 100K summary that lost to 5K.

## Where things live
- API entry: src/api/server.ts · routes in src/api/routes/
- Auth: src/auth/ (session.ts is the entry point)
- All database access goes through src/repo/ — never query directly

## Commands
- Test: npm test -- --silent   · Lint: npm run lint
- One package: npm test -w packages/<name>

## Landmines
- legacy/ is dead code; do not edit or import from it
- Migrations need a down-migration; the deploy skips forward-only ones silently

Measure your own retrieval#

The harness confound means you cannot inherit the answer. Four cheap measurements:

1. Read-utilisation. For 20 reads, what share of retrieved tokens appears in the final diff or explanation? Median under 20%: adopt symbol-level reads. Under 5%: retrieval is your main problem.
2. Read-coverage on failures. For your last 10 failures, was the decisive file ever read? This splits starvation (never read, a recall problem) from dilution (read and ignored, a precision problem). They have opposite fixes. Teams that skip this often apply the precision fix to a recall problem and make it worse.
3. Search efficiency. Turns until the first productive edit, and the ratio of search turns to edit turns. If search is over 60% of turns P, retrieval is the bottleneck.
4. A real A/B. Hold everything else fixed, vary only retrieval, run at least 20 tasks per arm, and report Pass@2 and Pass² — the full protocol is chapter 9's subject. Anything less is not evidence at these effect sizes.