docs+bench: audit follow-up — honesty caveat, opt-out docs, STATE anchor

A 3-agent triple-audit (fresh-clone health · adversarial dedup-math · session-arc
structure) verified the arc is healthy (1100/1100 from a clean clone, all
features smoke-verified, v4.2.0 live, no double-counting, P-model uncorrupted)
and surfaced real polish to land before the next step:

- BENCH HONESTY (HIGH, the W1.9-class catch): the dedup 99%/whole-session 91%
  numbers are a SAME-EPOCH best case — dedup doesn't fire after a context
  compaction (ADR-0003 reset), the most common real re-read trigger. Added the
  ⚠ same-epoch-ceiling caveat to the console + md + JSON, report the ACTUAL
  re-read rate (1/3 ≈ 33%, not the nominal 0.4 — Math.round artifact), and
  disclose dedup's share of the session baseline (~32%) so the lift over the
  P-model reads as a near-free bucket, not improved interception.
- ADR-0001 → ADR-0004 cross-ref: 0001 described a 'caller-file list'; 0004
  evolved it to call-site lines + the content/≥4-caller gates. Added the note.
- README: documented the per-feature env opt-outs (ENGRAM_GREP_INTERCEPT,
  ENGRAM_READ_DEDUP, ENGRAM_MISTAKE_GUARD) next to the kill switch — they were
  only in CHANGELOG/source before.
- docs/STATE.md: new single 'where we are / what's next' anchor (shipped arc,
  honest-claim through-line, ADR index, next-step candidates, gated items).

No code-behaviour change. Audit: tsc clean; full suite 1100/1100; bench runs
with the caveat; leak-scan 0 hits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: NickCirv

README.md

@@ -580,6 +580,18 @@ engram hook-enable     # removes the kill switch
 engram uninstall-hook  # surgical removal, preserves other hooks in settings.json
 ```
 
+**Per-feature opt-outs (env vars — disable one behaviour without the kill switch):**
+
+```bash
+ENGRAM_GREP_INTERCEPT=0   # don't answer symbol greps from the reference graph
+ENGRAM_READ_DEDUP=0       # don't dedup same-session re-reads of unchanged files
+ENGRAM_MISTAKE_GUARD=0    # don't warn before an edit that repeats a past mistake
+ENGRAM_NO_UPDATE_CHECK=1  # silence the passive "newer version available" hint
+```
+
+All interception is on by default and recall-safe; these let you turn off a single
+behaviour (e.g. in a hook config or shell profile) while keeping the rest.
+
 ---
 
 ## CLI Reference

bench/session-level.ts

@@ -441,9 +441,17 @@ async function main(): Promise<void> {
   const dedupBase = symbol.reduce((a, r) => a + r.dedupBaseline, 0);
   const dedupEng = symbol.reduce((a, r) => a + r.dedupEngram, 0);
   const dedupReduction = dedupBase > 0 ? ((dedupBase - dedupEng) / dedupBase) * 100 : 0;
+  // Actual re-reads per trace (Math.round, so e.g. R=0.4 over 3 reads = 1 ≈ 33%,
+  // NOT 40% — report the real fraction, not the nominal flag).
+  const reReadsPerTrace = Math.round(REPEAT_RATE * READS_PER_TRACE);
+  const actualRepeatPct =
+    READS_PER_TRACE > 0 ? (reReadsPerTrace / READS_PER_TRACE) * 100 : 0;
 
   // ── Whole session = first-reads/greps (P) + dedup (clean) ──
   const sessionBase = pBase + dedupBase;
+  // How much of the whole-session baseline is the (near-free) dedup bucket — so a
+  // reader knows the lift over the P-model is a cheap bucket, not better intercept.
+  const dedupShare = sessionBase > 0 ? (dedupBase / sessionBase) * 100 : 0;
 
   // Recall-coverage: of the match locations a raw content grep would show the
   // agent, what fraction does engram's packet actually surface? A bare file-list
@@ -515,15 +523,23 @@ async function main(): Promise<void> {
   console.log();
 
   console.log(
-    `Read dedup (re-reads at R=${REPEAT_RATE}): ${dedupReduction.toFixed(1)}% of the re-read budget ` +
-      `(${dedupBase} → ${Math.round(dedupEng)} tok).\n  A CLEAN saving — the re-read content is ` +
-      `already in the agent's context, so it is NOT discounted by P.`
+    `Read dedup (${reReadsPerTrace} re-read/trace of ${READS_PER_TRACE}, ~${actualRepeatPct.toFixed(0)}%): ` +
+      `${dedupReduction.toFixed(1)}% of the re-read budget (${dedupBase} → ${Math.round(dedupEng)} tok), ` +
+      `${dedupShare.toFixed(0)}% of the session baseline.`
+  );
+  console.log(
+    "  ⚠ SAME-EPOCH CEILING: dedup only fires when NO context compaction happened\n" +
+      "  since the first read (ADR-0003 resets the served-set on PreCompact /\n" +
+      "  SessionStart). A real re-read that FOLLOWS a compaction correctly re-serves,\n" +
+      "  so the realistic whole-session rate is BELOW these same-epoch figures."
   );
   console.log();
-  console.log("WHOLE SESSION (first-reads/greps at the P-model + dedup re-reads):");
+  console.log(
+    "WHOLE SESSION = first-reads/greps (P-model) + dedup re-reads — SAME-EPOCH CEILING:"
+  );
   console.log(
     `  P=0.00: ${sessionReductionAt(0).toFixed(1)}%  ·  P=0.50: ${sessionReductionAt(0.5).toFixed(1)}%  ` +
-      `(of ${sessionBase} session tokens)`
+      `(of ${sessionBase} session tokens; dedup is ${dedupShare.toFixed(0)}% of that base)`
   );
   console.log();
 
@@ -583,12 +599,19 @@ async function main(): Promise<void> {
           grepRawMatchLocations: rawLocs,
         },
         dedup: {
+          reReadsPerTrace,
+          actualRepeatPct: Number(actualRepeatPct.toFixed(1)),
           reReadBaselineTokens: dedupBase,
           reReadEngramTokens: dedupEng,
           reductionPct: Number(dedupReduction.toFixed(2)),
+          shareOfSessionBasePct: Number(dedupShare.toFixed(1)),
+          sameEpochCeiling: true,
+          caveat:
+            "Same-epoch best case — dedup does not fire after a context compaction (ADR-0003 reset); realistic rate is lower.",
         },
         wholeSession: {
           baselineTokens: sessionBase,
+          sameEpochCeiling: true,
           reductionAtP0: Number(sessionReductionAt(0).toFixed(2)),
           reductionAtP05: Number(sessionReductionAt(0.5).toFixed(2)),
         },
@@ -625,9 +648,11 @@ async function main(): Promise<void> {
     "",
     `## Read dedup (re-reads) + whole session`,
     "",
-    `**Read dedup: ${dedupReduction.toFixed(1)}%** of the re-read budget (${dedupBase} → ${Math.round(dedupEng)} tok) at a ${REPEAT_RATE} re-read rate (Phase-0 measured 38–46% of real reads are same-session repeats). A **clean** saving — the re-read content is already in the agent's context, so it is NOT discounted by P.`,
+    `**Read dedup: ${dedupReduction.toFixed(1)}%** of the re-read budget (${dedupBase} → ${Math.round(dedupEng)} tok), with ${reReadsPerTrace} re-read/trace of ${READS_PER_TRACE} (~${actualRepeatPct.toFixed(0)}%; Phase-0 measured 38–46% of real reads are same-session repeats). A **clean** saving — the re-read content is already in the agent's context, so it is NOT discounted by P.`,
+    "",
+    `> **⚠ Same-epoch ceiling.** Dedup only fires when no context compaction has happened since the first read — ADR-0003 resets the served-set on PreCompact / SessionStart. A real re-read that *follows* a compaction (the most common real trigger) correctly re-serves, so the realistic rate is **below** these figures. The dedup bucket is ${dedupShare.toFixed(0)}% of the whole-session baseline, so most of the lift over the P-model is this near-free bucket, not improved interception.`,
     "",
-    `**Whole session** (first-reads/greps at the P-model + dedup re-reads, ${sessionBase} tokens): P=0 **${sessionReductionAt(0).toFixed(1)}%** · P=0.5 **${sessionReductionAt(0.5).toFixed(1)}%**.`,
+    `**Whole session** (first-reads/greps at the P-model + dedup re-reads, ${sessionBase} tokens, same-epoch ceiling): P=0 **${sessionReductionAt(0).toFixed(1)}%** · P=0.5 **${sessionReductionAt(0.5).toFixed(1)}%**.`,
     "",
     `> The **P=0 number is an OPTIMISTIC ceiling**, not a lower bound: it assumes engram's structural packet fully answers what the agent grepped/read. The packet is a subset of the raw output, so the real saving lies between P=0 and break-even, by workload (high recall-sufficiency on structural/discovery work, low when exact call sites are needed — the same bimodal split as W1.9). Structural context-token reduction, **not** a bill saving. Traces generated by a rule (top-${TOP_SYMBOLS} most-referenced symbols × caller files), word-boundary grep baseline.`,
     "",

docs/STATE.md

@@ -0,0 +1,67 @@
+# engram — State & Next Step
+
+_Single source of truth for "where we are / what's next." Update at the end of each arc._
+
+**Last updated:** 2026-06-04 · **Live:** `engramx@4.2.0` "Loop" on npm · **main:** green (1100 tests, 0 vulns, tsc clean, fresh-clone verified)
+
+---
+
+## Where we are
+
+engram is a local code-graph context layer for AI coding tools. It indexes a repo into a SQLite
+knowledge graph once, then intercepts the agent's tool calls at the hook boundary and answers from
+the graph instead of letting raw file/grep output flood the context window.
+
+**The honest claim (do not drift):** every number engram reports is a *structural context-token
+reduction* — fewer tokens entering the model's context window per tool call — **not** a cost/bill
+saving. With prompt caching, engram's net effect on the dollar bill is ~0 (measured, W1.9). The value
+is capacity (longer sessions, fewer "context full" walls) and quality (ranked context, mistakes
+memory, audit). "Ranked" refers only to the **PageRank query ordering**, never the grep caller list.
+
+### Shipped this arc (v4.1 "Compass" → v4.2 "Loop")
+
+| Feature | What it does | Gated / recall-safe | ADR |
+|---|---|---|---|
+| Ranked context (PageRank) | Query results ranked over the `calls` reference graph | — | (v4.1) |
+| `callers/callees/impact` traversal | CLI/MCP over the reference graph | — | (v4.1) |
+| **Grep interception** | Content-mode symbol grep → **call-site lines** from the graph | `output_mode==="content"` + ≥4 caller files; `rg -n` escalation; `ENGRAM_GREP_INTERCEPT=0` | 0001, **0004** |
+| **Same-session read dedup** | Re-read of an unchanged file → pointer, not re-serve | byte-unchanged + full-read-only + PreCompact/SessionStart reset + TTL/cap; `ENGRAM_READ_DEDUP=0` | 0003 |
+| Day-1 mistakes | `fix:`/`fixes #N` miner seeds mistake-memory on init | `ENGRAM_MISTAKE_GUARD` (warn floor) | (v4.1) |
+
+### Measurement (how we keep ourselves honest)
+
+- `bench/real-world.ts` — per-file structural reduction (size-guarded "effective" number).
+- `bench/session-level.ts` — the session model: first-reads/greps via the **recall-recovery P-model**
+  (packets are lossy → reported as a curve over P + a break-even), plus a **recall-coverage** metric
+  (does the grep packet contain the actual usage lines), plus **dedup** re-reads as a separate *clean*
+  saving. The whole-session number is a **same-epoch ceiling** (dedup doesn't fire post-compaction) —
+  the bench discloses this; don't quote it as realistic.
+- `engram-counter@0.2.0` (separate public repo) — cache-aware cost accounting on real logs.
+
+ADR index: `docs/adr/0001`–`0004` (grep intercept · session bench · read dedup · richer find-usages).
+
+---
+
+## Open threads / next-step candidates
+
+1. **Bash-grep interception** — agents also run `rg`/`grep` via the **Bash** tool, which bypasses the
+   Grep-tool interception entirely. Closing it extends the v4.2 grep win to the Bash path. Real user
+   value, Phase-0-measurable (how often do agents grep via Bash?). _Strongest next lever._
+2. **git-bugfix-miner hardening** (task #70, deferred, non-blocking) — per-commit `git show` forks,
+   separator-byte parsing, surrogate slice. Perf/robustness, not correctness.
+3. **Workload router** — _parked._ Built on the W1.9 bimodal, our weakest evidence (N=3, high
+   variance). The per-call honest gates we already ship (read size-guard, grep content+caller gate,
+   dedup) are effectively a better router — they only fire when engram genuinely saves. Revisit only
+   if the bimodal firms up with more runs.
+
+## Gated (needs Nick / not engineering)
+
+- **LEAK-P2** — purge advisory docs + paths from OLD engram-counter history (git-filter-repo +
+  force-push). Drafted; awaiting authorization.
+
+## Working rhythm (what produced this arc)
+
+Phase-0 measure → build gated + recall-safe → triple-audit (tsc + full suite + e2e from built CLI) →
+**adversarial review** (every feature this arc had a real issue caught: dedup recall holes, the
+find-usages token regression, the bench inflation caveat) → leak-audit → ship. Don't skip the
+adversarial pass — it has paid for itself every single time.

docs/adr/0001-grep-symbol-intercept.md

@@ -1,6 +1,13 @@
 # ADR-0001: Intercept symbol-search Grep with the reference graph (recall-safe)
 
-**Status:** Accepted · **Date:** 2026-06-03 · **Author:** Nicholas
+**Status:** Accepted (evolved by [ADR-0004](0004-grep-richer-find-usages.md)) · **Date:** 2026-06-03 · **Author:** Nicholas
+
+> **Update (ADR-0004):** the deny payload no longer returns a bare *caller-file
+> list* (as described below) — it returns the actual **call-site lines**
+> (`file:line: code`) scanned from those files, and interception is gated to
+> `output_mode === "content"` + `≥4` caller files. The recall-safe design and the
+> `rg -n` escalation here still hold; only the packet contents and the gates
+> evolved. Read 0001 for the *why*, 0004 for the *what ships*.
 
 ## Context