release: v1.108.167 — cue-anchored delivery ledger (measure what we hand over twice)

The session counted byte-identical repeat calls and consumed the count
nowhere, and that counter was blind to the shape that actually costs: the
same symbol re-delivered under a different query. Two queries, two argument
hashes, one set of bytes paid for twice.

P0 (measure): new session-scoped delivery ledger reports redelivered_symbols,
redelivery_rate and redelivered_tokens_est in the yield block. The PRD carries
a binding pre-registered rule set before the number exists: suppression only
ships if the measured rate clears 10%.

P1 (annotate): advisory _meta.already_delivered {count, symbols}. No response
body changed and nothing is suppressed. Telling an agent it already holds the
bytes may fix most of it, and costs nothing to find out first.

note_served is deliberately untouched: its record is what finalize_handoff
attests evidence_refs against, so the ledger sits parallel to it rather than
broadening what a handoff can cite. Only full-source deliveries are priced;
edit-eviction keeps stale bytes from being announced as already-delivered.

Origin: Token Cost Radar 2026-07-24, arXiv 2607.20972. The paper's 39% figure
is theirs over their harness and is not claimed here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: jgravelle

CHANGELOG.md

@@ -2,6 +2,65 @@
 
 All notable changes to jcodemunch-mcp are documented here.
 
+## [1.108.167] - 2026-07-24 - cue-anchored delivery ledger: measure what we hand over twice
+
+### Added
+
+- **The session now measures how often it re-delivers a symbol it already
+  bought.** jcm counted byte-identical repeat calls (`note_call_signature` →
+  `yield.repeated_identical_calls`) and did nothing with the count, and that
+  counter could not see the shape that actually costs: the **same symbol
+  re-delivered under a different query**. `search_symbols("auth handler")` and
+  `get_ranked_context("who validates the token")` can return the same three
+  bodies at full byte cost with two different `args_hash` values — invisible.
+  New delivery ledger in `storage/token_tracker.py` (`_delivered`, capped at
+  `_DELIVERED_MAXSIZE = 5000`, insertion-ordered, process lifetime only, never
+  on disk) records `{count, tokens, full_source}` per symbol id.
+- **`yield` block gains `redelivered_symbols` / `redelivery_rate` /
+  `redelivered_tokens_est`** in `get_session_stats`. This is the P0
+  measurement, and it is the gate: a **pre-registered, binding** decision rule
+  (`docs/prd-cue-anchored-delivery.md` §3) says suppression only ships if the
+  measured rate clears 10%. Threshold set before the number exists, honored
+  after — the v1.108.149 cache-stability precedent, where the measurement was
+  the deliverable and the pre-registered threshold told us to hold.
+- **Advisory `_meta.already_delivered`** `{count, symbols[]}` when a response
+  carries symbols the session already received. **Annotation only — no response
+  body changes and nothing is suppressed.** Telling an agent "you already have
+  this" may fix most of it, and costs nothing to find out first. List capped at
+  `_DELIVERY_ANNOTATE_MAX = 20` (count stays exact) so a broad re-search can't
+  flood the envelope.
+- Recorded at the existing `call_tool` chokepoint via new
+  `server._delivery_entries`, covering `search_symbols`, `get_ranked_context`,
+  `get_symbol_source` and `get_context_bundle` (both the flat and batch shapes
+  of the shape-follows-input pair).
+
+### Notes
+
+- **`note_served` is deliberately untouched.** Its record is what the handoff
+  contract ([#374](https://github.com/jgravelle/jcodemunch-mcp/issues/374) /
+  [#377](https://github.com/jgravelle/jcodemunch-mcp/issues/377)) attests
+  `evidence_refs` against; broadening it to the source-dump tools would silently
+  change what a handoff can cite. The delivery ledger is a **parallel** record.
+  Likewise `_yield_served`'s value type is a followed-through bool the yield
+  block sums over, so the ledger sits beside it rather than extending it.
+- **Only full-source deliveries are priced.** A repeat of a signature/summary
+  row is reported but never accrues `redelivered_tokens_est` — re-showing a
+  signature is cheap. A symbol first seen as a search row and later fetched in
+  full is **new bytes, not a redelivery**.
+- **Edit-eviction is the invalidation that ships.** `note_edited_files` now also
+  drops ledger entries for touched files (Windows-separator and suffix-tolerant,
+  reusing the existing path matcher), so stale bytes are never annotated as
+  "you already have this". `content_hash` invalidation is a stated
+  **prerequisite for suppression**, not needed for annotation.
+- **Byte-identical for existing callers**: the `yield` block keeps its exact
+  prior shape when nothing was delivered, and no response body changed. NO new
+  tool, NO schema/tool-count/`INDEX_VERSION` change.
+- Origin: Token Cost Radar 2026-07-24 research watch (arXiv 2607.20972,
+  cue-anchored working memory). Clean-room; the paper's 39% intra-session reread
+  figure is **theirs over their harness** and is deliberately not claimed here —
+  we publish our own number or none. PRD:
+  `docs/prd-cue-anchored-delivery.md`. New `tests/test_v1_108_167.py` (33).
+
 ## [1.108.166] - 2026-07-24 - absence evidence: cite a zero-result scan as proof (handoff/v2 phase 3, #377)
 
 ### Added

CLAUDE.md

@@ -1,7 +1,8 @@
 # jcodemunch-mcp — Project Brief
 
 ## Current State
-- **Version:** 1.108.166 — **Absence evidence: a ZERO-RESULT scan is now citable proof (handoff/v2 phase 3, [#377](https://github.com/jgravelle/jcodemunch-mcp/issues/377), @mightydanp).** v1/v2 could not cite a zero-result search — nothing served ⇒ no id — yet "searched the complete, fresh, non-truncated index and it is NOT there" is the claim audits most need and agents most often assert unproven. **No new retrieval machinery:** `retrieval/verdict.build_verdict` already emits state/scanned/channels/coverage/scorer; `handoff.note_absence` records those verdicts under a deterministic ref. Flow mirrors symbol evidence — **server hands you the token, you cite it back**: an `absent` verdict now carries `_meta.verdict.evidence_ref`. **The REFUSAL RULES are the feature (his, adopted verbatim):** only `absent` proves absence; `low_confidence`/`degraded` do NOT (weak/partial scan ≠ evidence of nothing); **stale index** does NOT (describes an older tree); **truncated index** does NOT (target may sit in the dropped tail). A refused scan is STILL RECORDED so citing it returns the REASON (`refused_absence` / `refused_absence_claims` naming the claim), not a bare unknown-ref; and when absent-but-not-citable the live response says `absence_citable:false` + `absence_blocked_by` rather than handing out a token that fails later. **Rendered proof = the auditable part:** body carries tool+query, the SCOPE it wasn't found in, symbols/files scanned, channels, coverage w/ exclusion counts + generation SHA, scorer. Unknown coverage renders "not recorded for this index (scope unknown)" — NEVER as a complete scope. Detail renders ONCE (under its claim when it has one). Ref = sha256[:12] over `(tool, repo, query, scope)` — same scan+scope = same proof, narrowed scope = different proof. Session-scoped, in-memory, capped, never on disk. Server chokepoint in `call_tool` before encoding (one place, covers every verdict-emitting tool). New `tests/test_v1_108_166.py` (23, one per refusal rule); receipt gains `absence_attested` (omitted otherwise). NO new tool/schema/tool-count/INDEX_VERSION change; **core_compact unchanged 3996**; ruff clean; suite 5618. ⚠ **Suite parity NOT yet shipped:** jdoc ports cleanly (has `channels.index`) but its default `meta_fields` STRIPS `_meta`, so the ref must be re-attached AFTER filtering like the v1.104.0 budget block. **jdata has NO `index` channel — the stale gate cannot fire there; do NOT port silently (a safety rule that looks enforced and isn't). jjg to choose: disclose-as-untracked vs refuse.** #377 stays OPEN; P2 (evidence receipts) + P4 (requirement matching) still deferred.
+- **Version:** 1.108.167 — **Cue-anchored delivery ledger: the session now MEASURES what it hands over twice (P0+P1 of `docs/prd-cue-anchored-delivery.md`).** jcm counted byte-identical repeat calls (`note_call_signature` → `yield.repeated_identical_calls`) and consumed the count nowhere; worse, that counter is **blind to the shape that actually costs** — the SAME symbol re-delivered under a DIFFERENT query (two queries ⇒ two `args_hash` ⇒ invisible, one set of bytes paid twice). New ledger `_State._delivered` (`storage/token_tracker.py`): OrderedDict `sid -> {count, tokens, full_source}`, cap `_DELIVERED_MAXSIZE=5000`, process-lifetime, never on disk. **Deliberately PARALLEL to `_yield_served`, not an extension of it** — that map's value is a followed-through bool `_yield_locked()` sums over, and its key set is the record `finalize_handoff` attests `evidence_refs` against, so **`note_served` is UNTOUCHED** (broadening it to the source-dump tools would silently change what a handoff can cite — revisit as its own contract change, never as a side effect). **P0 = the measurement + the gate:** yield block gains `redelivered_symbols`/`redelivery_rate`/`redelivered_tokens_est`, and the PRD carries a **BINDING pre-registered rule** (<10% ⇒ suppression never ships; 10-25% ⇒ opt-in; >25% ⇒ reopen default) set BEFORE the number exists — the v1.108.149 cache-stability precedent where the measurement WAS the deliverable and the threshold said hold. **P1 = annotation only:** `_meta.already_delivered {count, symbols[:20]}` (count exact, list capped so a broad re-search can't flood the envelope); **no response body changed, nothing suppressed** — telling the agent "you already have this" may fix most of it and costs nothing to learn first. Recorded at the existing `call_tool` chokepoint via new `server._delivery_entries` (search_symbols / get_ranked_context / get_symbol_source / get_context_bundle, both flat + batch shapes). **Only full-source deliveries are priced** — a repeat signature row is reported, never charged; signature-then-full-source is NEW BYTES not a redelivery. **Invalidation that ships = edit-eviction** (`note_edited_files` now also DELETES ledger entries for touched files, reusing the existing Windows-separator/suffix-tolerant matcher) so stale bytes are never announced as already-delivered; **`content_hash` invalidation is a stated PREREQUISITE for P2 suppression**, not needed for annotation (and the chokepoint has no index handle anyway). Yield block byte-identical when nothing delivered. NO new tool/schema/tool-count/INDEX_VERSION change; ruff clean; suite 5645 + the known 12 local-ONNX semantic env fails. New `tests/test_v1_108_167.py` (33). **jjg deferred all 4 PRD open questions to me 2026-07-24; resolved in §7 — binding rule, opt-in forever (no default-on in 1.x), one ledger w/ `full_source` flag, P3 split out.** Origin: Radar 2026-07-24 arXiv 2607.20972; **the paper's 39% is THEIRS over THEIR harness — never claim it, publish ours or none.** ⚠ **P2/P3 NOT started and P2 is gated on P0's real number** — get a measurement from a replay corpus before writing any suppression code.
+- **Prior (1.108.166):** — **Absence evidence: a ZERO-RESULT scan is now citable proof (handoff/v2 phase 3, [#377](https://github.com/jgravelle/jcodemunch-mcp/issues/377), @mightydanp).** v1/v2 could not cite a zero-result search — nothing served ⇒ no id — yet "searched the complete, fresh, non-truncated index and it is NOT there" is the claim audits most need and agents most often assert unproven. **No new retrieval machinery:** `retrieval/verdict.build_verdict` already emits state/scanned/channels/coverage/scorer; `handoff.note_absence` records those verdicts under a deterministic ref. Flow mirrors symbol evidence — **server hands you the token, you cite it back**: an `absent` verdict now carries `_meta.verdict.evidence_ref`. **The REFUSAL RULES are the feature (his, adopted verbatim):** only `absent` proves absence; `low_confidence`/`degraded` do NOT (weak/partial scan ≠ evidence of nothing); **stale index** does NOT (describes an older tree); **truncated index** does NOT (target may sit in the dropped tail). A refused scan is STILL RECORDED so citing it returns the REASON (`refused_absence` / `refused_absence_claims` naming the claim), not a bare unknown-ref; and when absent-but-not-citable the live response says `absence_citable:false` + `absence_blocked_by` rather than handing out a token that fails later. **Rendered proof = the auditable part:** body carries tool+query, the SCOPE it wasn't found in, symbols/files scanned, channels, coverage w/ exclusion counts + generation SHA, scorer. Unknown coverage renders "not recorded for this index (scope unknown)" — NEVER as a complete scope. Detail renders ONCE (under its claim when it has one). Ref = sha256[:12] over `(tool, repo, query, scope)` — same scan+scope = same proof, narrowed scope = different proof. Session-scoped, in-memory, capped, never on disk. Server chokepoint in `call_tool` before encoding (one place, covers every verdict-emitting tool). New `tests/test_v1_108_166.py` (23, one per refusal rule); receipt gains `absence_attested` (omitted otherwise). NO new tool/schema/tool-count/INDEX_VERSION change; **core_compact unchanged 3996**; ruff clean; suite 5618. ⚠ **Suite parity NOT yet shipped:** jdoc ports cleanly (has `channels.index`) but its default `meta_fields` STRIPS `_meta`, so the ref must be re-attached AFTER filtering like the v1.104.0 budget block. **jdata has NO `index` channel — the stale gate cannot fire there; do NOT port silently (a safety rule that looks enforced and isn't). jjg to choose: disclose-as-untracked vs refuse.** #377 stays OPEN; P2 (evidence receipts) + P4 (requirement matching) still deferred.
 - **Prior (1.108.165):** **Claim-scoped evidence: a handoff section may carry caller-authored `claims`, each with its OWN `evidence_refs` (handoff/v2 phase 1, [#377](https://github.com/jgravelle/jcodemunch-mcp/issues/377), @mightydanp).** v1 proved a ref was retrieved this session but never bound it to a sentence — `_validate_evidence` attested against served ids/file components and `render_handoff` dumped every ref into ONE global block. Now `_validate_claims` (new) takes `{id, statement, evidence_refs, classification?}`; **ids unique across the WHOLE handoff, not per section** (the id is the citation anchor — two sections owning one id makes a citation ambiguous); statements/classifications preserved VERBATIM (server never authors); each claim's refs attested SEPARATELY through the unchanged `_validate_evidence`, so an unknown ref returns `invalid_claims: [{claim_id, unknown_refs}]` naming the claim instead of one global failure list. `render_handoff` prints `### <statement>` + `- Claim id:` + indented evidence, and now takes the schema string as a param. **Three calls the proposal left open:** (1) **the INPUT picks the contract** — no claims anywhere ⇒ schema stays `jcodemunch.handoff/v1`, body BYTE-IDENTICAL to v1, `claims_attested` omitted (not `0`); any claim ⇒ `.../v2`. (2) **claims can satisfy `evidence_refs`** (top-level may be empty when claims carry refs — strictly more permissive, no existing call changes). (3) **claim refs join the canonical index**, caller order first, so a v1 consumer reading a v2 handoff sees every ref where it expects. Section `content` optional ONLY when claims present. `finalize_handoff` is standard tier ⇒ **core_compact unchanged 3996**; standard/full +123 tok (baseline refreshed). New `tests/test_v1_108_165.py` (18, incl. the byte-identical-v1 guard). NO INDEX_VERSION/tool-count change. ⚠ **Known limit, disclosed publicly on #377 first:** phase 1 does NOT narrow what counts as a match — the file-component broadening in `_validate_evidence` means citing a whole file still attests when one unrelated symbol from it was served. That's phase 2 (evidence receipts), DEFERRED. Suite parity same day: jdoc v1.116.0 + jdata v1.25.0. **#377 stays OPEN as the arc tracker; phase 3 (absence evidence) is next and is the one we want most.**
 - **Prior (1.108.164):** — **A path-shaped `repo` arg returns a routed error instead of a raw storage crash (#376, split out of #375; reported from a daily Ubuntu install).** `tools/_utils.resolve_repo` split any `repo` containing `/` on the FIRST separator and returned the halves UNVALIDATED (`_utils.py:117`), so `repo="src/auth/service.ts"` → `owner="src"`, `name="auth/service.ts"`. Callers DO catch `ValueError` from the resolver (`search_symbols.py:626`), but the bad pair then reached `store.load_index` and raised `ValueError: Path separator in name` from `SQLiteIndexStore._safe_repo_component` (`sqlite_store.py:2881`) — **outside every handler**. Since `_safe_repo_component` rejects a separator in `name` at WRITE time, an `owner/name` id can never legitimately carry a second one, so **every two-slash `repo` argument was a guaranteed crash**, on every tool routing through the shared resolver (not just search_symbols). Reproduced on .163, not only the reporter's .158. Fix: a post-split name still containing `/` or `\` is treated as a path — routed to `_resolve_path_repo` first (which now also picks up bare relative MULTI-SEGMENT paths like `apps/web/src` that `_looks_like_path` is deliberately too conservative to match, v1.108.159's acceptance only covered `.`/`~`/backslash/absolute), else raises new `_path_shaped_repo_error` naming `resolve_repo` + `index_folder` + **`file_pattern=` when the arg has a file suffix** (the reporter's ask — an agent passing `repo=<file>` usually meant to scope the search). `_resolve_path_repo`'s unindexed-path raise routes through the same helper so both give one message. **Single-separator ids byte-identical** (`tools/agents.ts` still takes the id path → clean in-band not-loadable error). New `tests/test_v1_108_164.py` (8, incl. the single-slash no-regression guard). NO schema/tool-count/INDEX_VERSION change. Suite 5577 passed + the known 12 local-ONNX semantic env fails.
 - **Prior (1.108.163):** — **Audit P2 pair closed: WS-8 license-key transport (V12) + WS-7 benchmark integrity (V1).** **WS-8:** `org/license._check_server` now POSTs `{product, license}` as a form body (key NEVER in the URL → can't land in server/proxy access logs); `cli/install_pack` carries the key in the `X-JCM-License` header. Each has a ONE-SHOT legacy fallback keyed to the exact pre-deploy backend signature ("Missing license parameter." for validate.php POST; the no-license 403 "requires a jCodeMunch license" for packs) so ship order can't strand a paying customer — a REAL key rejection (not found/revoked/expired/"Invalid or expired") never retries. Backend CONFIRMED POST/header-aware on j.gravelle.us 2026-07-23 (live curl verification, all transports) — the two client fallbacks (`_needs_legacy_get_fallback`, `_looks_like_missing_license_response`) can be removed in a future release. Backend mirror EDITED (jjg deploys via SFTP): `C:\MCPs\web\jCodeMunch.com\validate.php` (POST form/JSON body + X-JCM-License header + legacy GET; Cache-Control: no-store) + `starter-packs-system/api/index.php` (`request_license()` helper on download/validate; internal `validate_license()` now POSTs to validate.php). NOTE: root `C:\MCPs\web\validate.php` copy is STALE (pre-jbuffer, May 16) — canonical mirror is `jCodeMunch.com\`. Bonus fix: install-pack's `raise_for_status()` rendered every 4xx JSON license/pack error as "Could not reach the starter packs server" — removed; JSON error branch now actually runs. **WS-7:** `benchmarks/results.md` REGENERATED at .163 against same-day GitHub re-indexes (express 172 / fastapi 1,000 / gin 109 files): **99.6% aggregate** (5,799,695→25,220 tok, 230x), per-query 99.1–99.9%; hand-written A/B sections preserved verbatim (regen→scratch→merge). README headline RECONCILED — the stale "95% average" table (old 34-file capped express index) replaced; README/results.md/METHODOLOGY.md/provenance measured.json all quote 99.6% now (provenance registry itself unchanged — it pins average_pct 99.6 which held; drift-guard test green). "95%+" floor phrasing elsewhere kept (conservative, true). Harness dead fallback fixed (`get_file_content_text` never existed → `store.get_file_content`). `benchmarks/token_baselines/v1.108.163.json` captured (in-process driver; NOTE `__version__` is "unknown" under bare PYTHONPATH — driver stamps version explicitly) → `analyze_perf(compare_release="1.108.163")` resolves. Replay CI gate was already wired (replay.yml, 2% gate) — WS-7 acceptance met. WS-8 CI siblings (SHA-pinned sign-release, dependabot github-actions) verified already done; `uv sync --locked` still deliberately parked (uv.lock drifted). Tests `tests/test_v1_108_163.py` (28 w/ params); suite 5569 passed + the known 12 local-ONNX semantic env fails. **Audit P2 queue now EMPTY — remaining audit work is P3s only.**