feat(intercept): richer find-usages in the Grep packet (ADR-0004)

The session bench showed the grep packet was a file list that contained the
investigated symbol ZERO times — high effective-P (low recall-sufficiency) that
caps the real saving. The handler now scans the resolved caller files at query
time (no graph/miner change) and returns the actual call-site lines
(file:line: code), deduplicated and capped, plus the rg -n escalation — so the
agent gets the usage context it grepped for and rarely re-greps.

An adversarial audit then caught that the naive version was a token REGRESSION
on most greps (bigger than a default filenames grep always; bigger than a
content grep for low-usage symbols, where ~50 tok of boilerplate dominates).
Two gates fix it so every interception that fires is a genuine win:
- output_mode === 'content' only (the default files_with_matches / count modes
  return filenames / a number — cheaper than any packet → passthrough).
- callerFiles.length >= 4 (below that a content grep is small enough that the
  packet would cost more → passthrough).
Measured on engram's own repo (content mode): init 573 vs 9,317 tok, parse 578
vs 2,152, getStore 531 vs 1,781 — all smaller; files-mode + low-usage → passthrough.

Also from the audit: word boundaries via lookarounds (so $-identifiers anchor),
and the cap note no longer claims a specific (wrong) omitted-file count. Caps:
15 caller files scanned, 25 lines, 140 chars/line, files > 1 MB skipped.
Best-effort, never throws; 0 call sites found (dynamic dispatch) → passthrough.

Audit: tsc clean; full suite 1100/1100 (+3 grep tests: call-site format, cap,
two gate passthroughs); e2e re-measured (content→smaller, files-mode→passthrough,
low-usage→passthrough); adversarial review SHIP-WITH-FIXES → all 3 fixes applied.
Leak-scan 0 hits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: NickCirv

CHANGELOG.md

@@ -21,9 +21,15 @@ All notable changes to engram are documented here. Format based on
 - **Grep interception (research-loop elimination — ADR-0001).** A new
   `PreToolUse:Grep` handler answers symbol-usage searches from the `calls`
   reference graph instead of letting a raw match dump flood the context window.
-  When the agent greps a bare identifier that's a known symbol with references,
-  engram denies the grep and returns the list of files that reference it,
-  plus the exact `rg -n "<pattern>"` escalation command. **Recall-safe by
+  For a **content-mode** grep of a well-referenced symbol (≥4 caller files),
+  engram denies the grep and returns the actual **call-site lines**
+  (`file:line: code`, scanned from the resolved caller files, deduplicated and
+  capped — ADR-0004) plus the exact `rg -n "<pattern>"` escalation command.
+  The call-site lines give the agent the usage context it grepped for (so it
+  rarely re-greps) while staying smaller than the content grep it replaces — e.g.
+  on engram's own repo `init` is 573 tok vs 9,317 for the raw grep. Default
+  `files_with_matches` / `count` greps and low-usage symbols pass through (their
+  grep is already cheaper than any packet). **Recall-safe by
   construction:** regex/text/multi-word/stopword patterns and unknown symbols
   pass straight through (grep out-recalls the structural graph there), and the
   escalation command means the agent can always recover full textual matches in

docs/adr/0004-grep-richer-find-usages.md

@@ -0,0 +1,53 @@
+# ADR-0004: Richer find-usages in the Grep packet (call-site lines)
+
+**Status:** Accepted · **Date:** 2026-06-03 · **Author:** Nicholas
+
+## Context
+
+ADR-0001 made the Grep handler answer a symbol search from the `calls` graph with a list of caller
+**files**. The session-level bench (ADR-0002) and its adversarial audit then showed the load-bearing
+weakness: that packet is a strict *subset* of what the agent grepped for — it contains the symbol's
+call sites **zero times**, only file names. So the agent often re-runs the raw grep to see the actual
+usage, which is exactly the high "recall-recovery fraction" (P) that caps the real session saving
+between the optimistic ceiling and break-even. To move the real saving toward the ceiling we must raise
+**recall-sufficiency**: make the packet actually answer "where/how is this symbol used."
+
+## Decision
+
+The Grep handler now scans the resolved caller files (at query time, no graph/miner change) for the
+lines that reference the symbol (word-boundary via lookarounds, so `$`-identifiers anchor) and returns
+the actual **call-site lines** (`file:line: code`) in the packet, deduplicated and capped, plus the
+existing `rg -n` escalation.
+
+**Two gates make this a genuine token win, never a regression** (added after an adversarial audit showed
+the naive version cost *more* on most greps):
+
+1. **`output_mode === "content"` only.** Claude Code's Grep defaults to `files_with_matches` (filenames)
+   and also offers `count` — both far cheaper than any packet. engram's call-site packet can only beat a
+   *content* grep (matching lines), so for the filenames/count modes it passes through.
+2. **`callerFiles.length >= 4`.** Below that, even a content grep is small enough that engram's packet +
+   its ~50-token boilerplate would cost more than the grep it replaces. The win scales with usage; this
+   floors out the regressing tail.
+
+Bounds keep the packet smaller than the content grep it replaces: at most 15 caller files scanned, 25
+lines returned, each trimmed to 140 chars, files > 1 MB skipped. If the scan finds **zero** lines (a
+`calls` edge from dynamic dispatch the literal symbol word doesn't appear for), it passes through.
+
+## The trade-off (honest)
+
+The packet is larger than a bare file list — but it is the right kind of larger: it contains the real
+usage context the agent grepped for, drawn **only from the files the graph says actually reference the
+symbol** (no comment/string/test-file/partial-match noise), capped. The naive version (no gates) was a
+token *regression* on most greps — bigger than a default filenames grep always, and bigger than a
+content grep for low-usage symbols (the ~50-token boilerplate dominates). The two gates fix that: scoped
+to `content` mode and `>= 4` caller files, every interception that fires is smaller than the content
+grep it replaces (measured on engram's own repo: `init` 573 vs 9,317 tok, `parse` 578 vs 2,152,
+`getStore` 531 vs 1,781), while raising recall-sufficiency from ~0 (file names) to the real call sites.
+This is a *structural* context-token effect, not a bill saving. The discarded alternative — returning every match
+line — is just the grep dump (no saving); the discarded alternative of reading the call-site line from
+`GraphEdge.sourceLocation` was rejected to avoid touching the hardened reference-graph code (the
+line *text* requires reading the file anyway, so the scan is simpler and self-contained).
+
+We cannot deterministically measure the drop in P (it is behavioural). We can and do measure the two
+proxies: (a) the packet still smaller than the raw grep, and (b) the packet now *contains* the call-site
+lines a grep would show (recall-coverage), vs ~0 before.

src/intercept/handlers/grep.ts

@@ -16,17 +16,31 @@
  * real grep runs. Default to doing nothing; act only when provably helpful.
  *
  * Returns:
- *   - A deny response whose reason is engram's caller list (the files that
- *     reference the symbol), when the pattern is a known symbol with references.
+ *   - A deny response whose reason is engram's call-site list (the actual
+ *     `file:line: code` lines that reference the symbol, from the files that
+ *     call it), when the pattern is a known symbol with references.
  *   - PASSTHROUGH (null) otherwise — caller writes nothing, exits 0, the real
  *     Grep runs unchanged.
  * Never throws. Every error path resolves to PASSTHROUGH via wrapSafely.
  */
+import { readFileSync, statSync } from "node:fs";
+import { join } from "node:path";
 import { PASSTHROUGH, isHookDisabled, type HandlerResult } from "../safety.js";
 import { findProjectRoot, isValidCwd } from "../context.js";
 import { buildDenyResponse } from "../formatter.js";
 import { callers } from "../../core.js";
 
+/** Bounds that keep the packet richer than a file list yet smaller than a raw
+ *  repo-wide grep (ADR-0004). */
+const MAX_CALLER_FILES = 15;
+const MAX_SITES = 25;
+const MAX_LINE_LEN = 140;
+const MAX_FILE_BYTES = 1_000_000;
+/** Below this many caller files a content grep is small enough that engram's
+ *  packet + boilerplate would cost MORE than the grep it replaces — so pass
+ *  through. The win scales with usage; this floors out the regressing tail. */
+const MIN_CALLER_FILES = 4;
+
 export interface GrepHookPayload {
   tool_name?: string;
   cwd?: string;
@@ -80,6 +94,12 @@ export async function handleGrep(
     return PASSTHROUGH;
   }
 
+  // (1b) Output-mode gate (ADR-0004): engram's call-site packet can only beat a
+  // CONTENT grep (matching lines). The default `files_with_matches` and `count`
+  // modes return just filenames / a number — cheaper than any packet — so for
+  // those, pass through and let the agent's cheap grep run.
+  if (payload.tool_input?.output_mode !== "content") return PASSTHROUGH;
+
   // (2) Resolve the project root from cwd (no file path on a Grep).
   const cwd = payload.cwd;
   if (typeof cwd !== "string" || !isValidCwd(cwd)) return PASSTHROUGH;
@@ -89,28 +109,112 @@ export async function handleGrep(
   // (3) Kill switch.
   if (isHookDisabled(projectRoot)) return PASSTHROUGH;
 
-  // (4) Ask the reference graph who references this symbol. No callers means
-  // either an unknown symbol or a textual-only occurrence — let grep run.
+  // (4) Ask the reference graph who references this symbol. Fewer than
+  // MIN_CALLER_FILES → a content grep would be small enough that engram's packet
+  // would cost more; pass through. (0 callers = unknown / textual-only symbol.)
   const callerFiles = await callers(projectRoot, pattern);
-  if (callerFiles.length === 0) return PASSTHROUGH;
+  if (callerFiles.length < MIN_CALLER_FILES) return PASSTHROUGH;
+
+  // (5) Pull the actual call-site lines from those files (ADR-0004) so the
+  // packet answers "where/how is it used", not just "which files". If the scan
+  // finds nothing (e.g. the edge came from dynamic dispatch the literal symbol
+  // word doesn't appear), let the agent's grep run.
+  const found = collectCallSites(projectRoot, callerFiles, pattern);
+  if (found.sites.length === 0) return PASSTHROUGH;
+
+  return buildDenyResponse(buildGrepAnswer(pattern, callerFiles.length, found));
+}
+
+interface CallSite {
+  readonly file: string;
+  readonly line: number;
+  readonly text: string;
+}
+interface CallSiteScan {
+  readonly sites: readonly CallSite[];
+  readonly filesOmitted: number;
+  readonly truncated: boolean;
+}
+
+function escapeRegExp(s: string): string {
+  return s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+}
 
-  return buildDenyResponse(buildGrepAnswer(pattern, callerFiles));
+/**
+ * Scan the resolved caller files for the lines that reference `symbol`
+ * (word-boundary) — the actual call sites the agent grepped for (ADR-0004).
+ * Bounded so the packet stays smaller than a raw grep: at most MAX_CALLER_FILES
+ * files, MAX_SITES lines, each trimmed to MAX_LINE_LEN; files > MAX_FILE_BYTES
+ * are skipped. Best-effort — unreadable files are skipped, never throws.
+ */
+function collectCallSites(
+  projectRoot: string,
+  callerFiles: readonly string[],
+  symbol: string
+): CallSiteScan {
+  // Identifier boundaries via lookarounds (not `\b`) so symbols containing `$`
+  // (jQuery/legacy) anchor correctly — `\b` can't sit next to a non-word `$`.
+  const re = new RegExp(`(?<![\\w$])${escapeRegExp(symbol)}(?![\\w$])`);
+  const sites: CallSite[] = [];
+  const scanFiles = callerFiles.slice(0, MAX_CALLER_FILES);
+  const filesOmitted = callerFiles.length - scanFiles.length;
+  let truncated = false;
+
+  for (const rel of scanFiles) {
+    if (sites.length >= MAX_SITES) {
+      truncated = true;
+      break;
+    }
+    let content: string;
+    try {
+      if (statSync(join(projectRoot, rel)).size > MAX_FILE_BYTES) continue;
+      content = readFileSync(join(projectRoot, rel), "utf-8");
+    } catch {
+      continue;
+    }
+    const lines = content.split("\n");
+    for (let i = 0; i < lines.length; i++) {
+      if (sites.length >= MAX_SITES) {
+        truncated = true;
+        break;
+      }
+      if (!re.test(lines[i])) continue;
+      let text = lines[i].trim();
+      if (text.length > MAX_LINE_LEN) text = text.slice(0, MAX_LINE_LEN) + "…";
+      sites.push({ file: rel, line: i + 1, text });
+    }
+  }
+  return { sites, filesOmitted, truncated };
 }
 
 /**
- * Format engram's structural answer plus the explicit escalation path. The
+ * Render engram's call-site answer plus the explicit escalation path. The
  * escalation line is non-negotiable — it's what makes denying the grep
  * recall-safe (the agent can always get the full textual matches).
  */
-function buildGrepAnswer(pattern: string, callerFiles: string[]): string {
-  const list = callerFiles.map((f) => `  - ${f}`).join("\n");
+function buildGrepAnswer(
+  symbol: string,
+  callerCount: number,
+  scan: CallSiteScan
+): string {
+  const lines = scan.sites.map((s) => `  ${s.file}:${s.line}: ${s.text}`);
+  const head =
+    `[engram] "${symbol}" — ${scan.sites.length}${scan.truncated ? "+" : ""} ` +
+    `call site(s) across ${callerCount} file(s) (resolved \`calls\` edges):`;
+  // When capped, don't claim a specific omitted-file count — the line cap can
+  // hit before all caller files are scanned, so any number would be wrong. Just
+  // point at the rg escalation for the complete set.
+  const capNote =
+    scan.truncated || scan.filesOmitted > 0
+      ? [`  … more call sites exist — run the rg command below for the complete set.`]
+      : [];
   return [
-    `[engram] "${pattern}" is referenced by ${callerFiles.length} file(s) ` +
-      `in the reference graph (structural \`calls\` edges):`,
-    list,
+    head,
+    ...lines,
+    ...capNote,
     "",
-    "This is engram's structural answer — resolved function/class references " +
-      "only. It does NOT include comments, strings, or dynamic references. " +
-      `If you need full textual matches, run: rg -n "${pattern}"`,
+    "This is engram's structural answer — resolved references in files that " +
+      "call the symbol. It may omit comments, strings, and dynamic references. " +
+      `For a full textual search, run: rg -n "${symbol}"`,
   ].join("\n");
 }

tests/intercept/handlers/grep.test.ts

@@ -52,18 +52,32 @@ export function lookup(name: string): number {
 }
 `
     );
+    // 3 more caller files so hashToken clears MIN_CALLER_FILES (≥4 callers →
+    // a content grep is big enough that engram's packet is a real win).
+    for (const n of [2, 3, 4]) {
+      writeFileSync(
+        join(projectRoot, "src", `svc${n}.ts`),
+        `import { hashToken } from "./util";\nexport function use${n}(x: string): number { return hashToken(x); }\n`
+      );
+    }
     await init(projectRoot);
   });
 
   afterEach(() => {
     rmSync(projectRoot, { recursive: true, force: true });
   });
 
-  function grepPayload(pattern: string): Record<string, unknown> {
+  function grepPayload(
+    pattern: string,
+    outputMode: string | undefined = "content"
+  ): Record<string, unknown> {
     return {
       tool_name: "Grep",
       cwd: projectRoot,
-      tool_input: { pattern },
+      tool_input:
+        outputMode === undefined
+          ? { pattern }
+          : { pattern, output_mode: outputMode },
     };
   }
 
@@ -127,7 +141,58 @@ export function lookup(name: string): number {
     // engram's structural answer names the calling file...
     expect(reason).toContain("svc.ts");
     expect(reason).toContain("hashToken");
+    // ...with the ACTUAL call-site lines (file:line: code) — the recall
+    // sufficiency a bare file list lacked (ADR-0004)...
+    expect(reason).toContain("call site");
+    expect(reason).toMatch(/src\/svc\.ts:\d+:/); // file:line prefix
+    expect(reason).toContain("return hashToken(name)"); // a real usage line
     // ...and ALWAYS includes the recall-safety escalation command.
     expect(reason).toContain('rg -n "hashToken"');
   });
+
+  it("caps the call-site list for a heavily-used symbol", async () => {
+    // a caller file with far more than MAX_SITES (25) references to hashToken
+    const many = Array.from(
+      { length: 60 },
+      (_, i) => `export function use${i}(n: string): number { return hashToken(n) + ${i}; }`
+    ).join("\n");
+    writeFileSync(join(projectRoot, "src", "heavy.ts"), `import { hashToken } from "./util";\n${many}\n`);
+    await init(projectRoot);
+
+    const r = (await handleGrep(grepPayload("hashToken"))) as Record<
+      string,
+      unknown
+    >;
+    const reason = (r.hookSpecificOutput as Record<string, unknown>)
+      .permissionDecisionReason as string;
+    // cap marker present, and the rendered line count never exceeds the cap
+    expect(reason).toContain("more call sites exist");
+    expect(reason).toMatch(/\d+\+ call site/); // "25+ call site(s)"
+    const siteLines = reason.split("\n").filter((l) => /:\d+:/.test(l));
+    expect(siteLines.length).toBeLessThanOrEqual(25);
+  });
+
+  it("passes through a non-content grep (files_with_matches / default / count)", async () => {
+    // engram's call-site packet can't beat a filenames-only or count grep
+    expect(await handleGrep(grepPayload("hashToken", "files_with_matches"))).toBe(
+      PASSTHROUGH
+    );
+    expect(await handleGrep(grepPayload("hashToken", "count"))).toBe(PASSTHROUGH);
+    // output_mode omitted entirely = default files mode → passthrough.
+    // (Built as a raw payload; passing `undefined` to grepPayload would trigger
+    // its `= "content"` default — the classic JS default-param gotcha.)
+    expect(
+      await handleGrep({
+        tool_name: "Grep",
+        cwd: projectRoot,
+        tool_input: { pattern: "hashToken" },
+      } as never)
+    ).toBe(PASSTHROUGH);
+  });
+
+  it("passes through a symbol with too few caller files (content grep is small)", async () => {
+    // `normalize` is called only by svc.ts (1 caller) < MIN_CALLER_FILES → a
+    // content grep would be tiny, so engram's packet would cost more → passthrough
+    expect(await handleGrep(grepPayload("normalize"))).toBe(PASSTHROUGH);
+  });
 });