release: v1.108.138 — confidence provenance: every number states its basis

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: jgravelle

CHANGELOG.md

@@ -2,6 +2,35 @@
 
 All notable changes to jcodemunch-mcp are documented here.
 
+## [1.108.138] - 2026-07-18 - confidence provenance: every number states its basis
+
+### Added
+- **Confidence provenance registry** (`retrieval/provenance.py`). Every
+  confidence constant the suite emits now traces to a stated basis:
+  `measured` — backed by a committed, reproducible benchmark artifact
+  (`benchmarks/provenance/measured.json`: the tiktoken token-reduction
+  methodology and the CI-gated replay retrieval-quality golden) — or
+  `declared` — an engineering prior, honestly labeled as exactly that. The
+  contract's load-bearing rule: **a prior is never presented as a
+  measurement.** A `declared` value graduates to `measured` only when a
+  gold-labeled corpus backs it, and `tests/test_provenance.py` fails the
+  build otherwise. Drift guards run in three directions: registry values
+  must equal the live constants in the emitting modules, registry `measured`
+  entries must equal the committed artifact, and the artifact must equal the
+  underlying benchmark results it cites (replay golden, methodology doc) —
+  no number can silently diverge from its receipt.
+- **`find_implementations` responses carry `_meta.confidence_provenance`** —
+  the per-channel basis for its resolution-confidence tiers (LSP/SCIP/AST/
+  duck-typed/decorator), so a caller can see that today's tiers are declared
+  priors, not measurements dressed up as ones.
+- **Published response JSON Schemas** in `schemas/`:
+  `retrieval-verdict.schema.json` (the unified `_meta.verdict` contract),
+  `confidence-provenance.schema.json`, and
+  `ranked-context-response.schema.json` (the full `get_ranked_context` JSON
+  surface, including `match_channel`/`_meta.query_shape` from v1.108.137).
+  CI validates live responses against them; external pipelines and agents
+  can do the same.
+
 ## [1.108.137] - 2026-07-18 - source-shaped exact seeding in ranked context
 
 ### Added

CLAUDE.md

@@ -1,6 +1,7 @@
 # jcodemunch-mcp — Project Brief
 
 ## Current State
+- **Version:** 1.108.138 — **Confidence provenance, Phase A of the self-attesting retrieval contract (F-15 #1 from the 2026-07-18 sweep; competitor unnamed in shipped artifacts).** New `retrieval/provenance.py`: `CONFIDENCE_PROVENANCE` registry — every confidence constant carries `basis` ∈ {`declared` (engineering prior, honestly labeled), `measured` (backed by committed artifact `benchmarks/provenance/measured.json`)}. Load-bearing rule: **a prior is NEVER presented as a measurement** — Phase B/C (graduating find_implementations tiers to measured) requires building a gold corpus FIRST (Loc-Bench feeds this); jjg accepted this phased order after my reservations (measurement-validity, fixture-decay drift-guard policy, jMRI-obligation optics). Registry entries: find_implementations 5 channels (all declared), negative_evidence_threshold 0.5, exact_seed_verdict_floor 1.0. `MEASURED` = the two artifacts jcm actually has: token_reduction 99.6%/15 runs/cl100k_base (METHODOLOGY.md) + replay retrieval quality (self_v1_75_0 golden, ndcg/mrr/recall 1.0, CI-gated). ⚠ The 0.992/0.895 Hit@10/MRR figure floated during the sweep is OCTOCODE'S benchmark (versus.php grid cell), NOT ours — never cite it as jcm's. `find_implementations` attaches `_meta.confidence_provenance` (via `channel_provenance()`). Published JSON Schemas in `schemas/` (retrieval-verdict, confidence-provenance, ranked-context-response); `tests/test_provenance.py` (14) drift-guards registry↔code↔artifact↔underlying-benchmark in all directions + jsonschema-validates live responses. README gains "Confidence provenance" subsection. NO new tool / INDEX_VERSION / schema-budget change (JSON-only `_meta` addition; no custom encoder for find_implementations so compact path unaffected). **Phase B next: surface measured munch-bench/methodology numbers as in-band provenance; Phase C: gold corpus per channel; jMRI normative section LAST (SHOULD-with-conformance-path, after suite parity is demonstrable).**
 - **Version:** 1.108.137 — **Source-shaped exact seeding in `get_ranked_context` (clean-room F-15; competitor unnamed in shipped artifacts).** New `retrieval/query_shape.py` classifies query tokens into identifier shapes (qualified `A::B`/`a.b` w/ file-extension-tail exclusion, CamelCase, snake_case, dunders; ≤3 tokens, prose never matches); the DEFAULT ranking path resolves each shaped token via a cached `name_map` (built beside the BM25 corpus) to exact-name symbols (case-sensitive→insensitive; qualified parent narrows via id/parent substring; PageRank ranks ties; include_kinds/scope honored; caps 3/token, 5/query) and PINS them ahead of the ranked tail (`_exact_seed_symbols` + pinning block after `scored.sort`). Items carry `match_channel:"exact_name"`; `_meta.query_shape={source_shaped,exact_seeded}`; verdict best_score floored at 1.0 (raw-BM25 scale vs 0.5 threshold) when seeded, and a seeded result bypasses the negative-evidence early return (`if not raw_scores and not seeded`). **Pure-prose queries byte-identical; `fusion=True` path unchanged (its WRR identity channel already covers exact).** Motivation: the exact-lookup lane is our one measured weak spot (definition MRR 0.70 vs 0.87 in the fair rerun). ALSO: (1) **`openWorldHint` on every tool** — new `_OPEN_WORLD_TOOLS` frozenset (index_repo, index_folder/index_file/summarize_repo [cloud summarizer opt-in], embed_repo/check_embedding_drift/test_summarizer, order/route front door) annotated True, everything else False, in `_apply_readonly_annotations`; machine-checkable no-network claim beside readOnlyHint. (2) **`PRAGMA journal_size_limit=64MB`** in sqlite_store `_PRAGMAS` + embedding_store + parse_cache (WAL starvation under long-lived readers — a failure class three ecosystem peers patched the same month). New `tests/test_v1_108_137.py` (19). NO INDEX_VERSION / tool-count / schema change. Origin: 2026-07-18 top-15 competitor sweep (memory `project_competitor_sweep_2026_07_18`); F-15 #1 (measured-provenance receipts / self-attesting contract) NOT started.
 - **Version:** 1.108.136 — **The savings meter records a per-day rollup (`daily` map in `_savings.json`).** The lifetime meter is the authoritative savings record but stored ONE number, so windowed views fell back to transcript scans — which on jjg's install prove ~2.2M tokens all-time vs the meter's 34.3B (4 orders of magnitude; jjg on the resulting Console panel: "\$7.73 this month next to \$171K = unhealthy disconnect", "\$0.11 today. Preposterous"). `_flush_locked` (`storage/token_tracker.py`) now also credits each flush batch to `daily[<local-iso-date>]` alongside `total_tokens_saved` — same file, same read-modify-write chokepoint (multi-process safe), `_DAILY_MAX_DAYS=750` cap, corrupt-shape tolerated, batch-across-midnight mis-dates ≤1 batch. **History accrues only from first flush under this version** — the lifetime total can't be back-distributed. **Long-running MCP servers keep pre-change code until restarted (editable install: new process = new code).** Consumer: console v0.8.81 draws window tiles from this book when it FULLY covers the window (today/yesterday immediately; week/month/year flip over as history accrues; `all` = lifetime total, closing the disconnect), transcript fallback + honest note otherwise; receipt scan remains sole source for per-tool breakdown + call counts. New `tests/test_v1_108_136.py` (6). NO INDEX_VERSION / wire change; file already README-disclosed.
 - **Version:** 1.108.135 — **`receipt --rates` publishes the model price table (kills the duplicate-constant drift class).** The rate table lived only inside `receipt`, so any consumer pricing its own token counts kept a COPY — and a copy drifts silently: the jMunch Console's duplicate sat at the retired **\$15** Opus rate long after this table moved to **\$5**, so its two dollar tiles disagreed 3x with nothing to catch it. New `--rates` prints `{rates_usd_per_mtok, default_model}` as JSON and exits; **scans no transcripts** (instant, cacheable) so a consumer can poll it cheaply and offer every priced model — including ones added later — with no code change on its side. `--model`'s choices already derive from the same table (.131), so CLI and published rates can't disagree. New `render_rates()`; forwarded through the `server.py` receipt subparser (the duplicate-parser surface — edit BOTH). +3 tests in `tests/test_v1_108_134.py` (incl. a guard that `--rates` never calls `iter_calls`). NO INDEX_VERSION / tool-count / wire change. Consumer: console v0.8.80 pricing-model picker.

README.md

@@ -128,9 +128,9 @@ is a byte the agent doesn't pay to read.
 <!-- WHATSNEW:START -->
 #### What's new
 
+- **[v1.108.138](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.138)** (2026-07-18) — confidence provenance: every number states its basis
 - **[v1.108.137](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.137)** (2026-07-18) — source-shaped exact seeding in ranked context
 - **[v1.108.136](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.136)** (2026-07-17) — the savings meter records a per-day rollup
-- **[v1.108.135](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.135)** (2026-07-17) — `receipt --rates` publishes the model price table
 <!-- WHATSNEW:END -->
 
 ![License](https://img.shields.io/badge/license-dual--use-blue)
@@ -325,6 +325,10 @@ The `tune_weights` tool reads the persistent ranking ledger and learns per-repo
 
 The `suggest_corrections` tool (and the `reflect` CLI) close the loop: they mine the same ranking ledger for **retrieval regret** — where retrieval failed and the agent had to re-ask — and return a prioritized, explainable set of *suggested* fixes (a CLAUDE.md routing or glossary line as a unified-diff preview, an index-freshness hint, a stale-config finding, a dry-run weight proposal). It is read-only by design: it suggests a patch and shows you the diff; applying it is your keystroke, never the server's. Requires `perf_telemetry_enabled` (it has a ledger to read only then) and returns an honest hint when off.
 
+### Confidence provenance — every number states its basis
+
+Every confidence constant the suite emits traces to a stated basis: **`measured`** (backed by a committed, reproducible benchmark artifact — `benchmarks/provenance/measured.json`, drift-guarded in CI so the constants and the artifact can never silently diverge) or **`declared`** (an engineering prior, honestly labeled as exactly that). `find_implementations` responses carry the per-channel basis in `_meta.confidence_provenance`, and the response contracts themselves are published as JSON Schemas in [`schemas/`](schemas/) (`retrieval-verdict`, `confidence-provenance`, `ranked-context-response`) so CI pipelines and agents can validate responses mechanically. A prior is never presented as a measurement: a `declared` value graduates to `measured` only when a gold-labeled corpus backs it, and a build that claims otherwise fails.
+
 ### Local-first speed
 
 Indexes are stored locally for fast repeated access.

benchmarks/provenance/measured.json

@@ -0,0 +1,25 @@
+{
+  "comment": "Committed measurement artifacts backing every basis='measured' entry in src/jcodemunch_mcp/retrieval/provenance.py. tests/test_provenance.py fails if the registry and this file diverge, and if this file diverges from the underlying artifacts it cites. Never hand-edit a number here without re-running the cited benchmark.",
+  "token_reduction": {
+    "average_pct": 99.6,
+    "task_runs": 15,
+    "baseline_tokens": 5122105,
+    "jcodemunch_tokens": 19406,
+    "tokenizer": "cl100k_base",
+    "run_date": "2026-03-28",
+    "methodology": "benchmarks/METHODOLOGY.md",
+    "results": "benchmarks/results.md"
+  },
+  "replay_retrieval_quality": {
+    "fixture": "self_v1_75_0",
+    "k": 10,
+    "queries": 10,
+    "ndcg": 1.0,
+    "mrr": 1.0,
+    "recall": 1.0,
+    "captured_at": "2026-07-06T01:03:32+00:00",
+    "generator_version": "1.108.99",
+    "source": "benchmarks/replay/results/self_v1_75_0-golden.json",
+    "ci_gated": true
+  }
+}

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "jcodemunch-mcp"
-version = "1.108.137"
+version = "1.108.138"
 description = "Token-efficient MCP server for source code exploration via tree-sitter AST parsing"
 readme = "README.md"
 requires-python = ">=3.10"

schemas/confidence-provenance.schema.json

@@ -0,0 +1,29 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://jcodemunch.com/schemas/confidence-provenance.schema.json",
+  "title": "jCodeMunch confidence provenance",
+  "description": "The _meta.confidence_provenance block: every confidence constant a tool emits states its basis. 'declared' = an engineering prior, honestly labeled; 'measured' = backed by a committed benchmark artifact (benchmarks/provenance/measured.json), drift-guarded in CI. A declared value graduates to measured only when a gold-labeled corpus exists for it — a prior is never presented as a measurement.",
+  "type": "object",
+  "required": ["channels", "contract"],
+  "properties": {
+    "channels": {
+      "type": "object",
+      "minProperties": 1,
+      "additionalProperties": {
+        "type": "object",
+        "required": ["value", "basis"],
+        "properties": {
+          "value": {"type": "number", "minimum": 0, "maximum": 1},
+          "basis": {"enum": ["declared", "measured"]},
+          "source": {
+            "type": "string",
+            "description": "Repo-relative path of the committed measurement artifact; required in spirit when basis=measured."
+          }
+        },
+        "additionalProperties": true
+      }
+    },
+    "contract": {"type": "string"}
+  },
+  "additionalProperties": false
+}

schemas/ranked-context-response.schema.json

@@ -0,0 +1,83 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://jcodemunch.com/schemas/ranked-context-response.schema.json",
+  "title": "jCodeMunch get_ranked_context response",
+  "description": "JSON response contract for get_ranked_context (the tool's JSON surface; the MUNCH compact encoding is a lossless-scalar/lossy-detail projection of this shape). The _meta.verdict block additionally conforms to retrieval-verdict.schema.json.",
+  "type": "object",
+  "required": ["context_items", "total_tokens", "budget_tokens", "items_included", "items_considered", "_meta"],
+  "properties": {
+    "context_items": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["symbol_id", "relevance_score", "centrality_score", "combined_score", "tokens", "source"],
+        "properties": {
+          "symbol_id": {"type": "string"},
+          "match_channel": {
+            "enum": ["exact_name"],
+            "description": "Present when the item was pinned by source-shaped exact seeding rather than ranked scoring."
+          },
+          "relevance_score": {"type": "number", "minimum": 0, "maximum": 1},
+          "centrality_score": {"type": "number", "minimum": 0, "maximum": 1},
+          "combined_score": {"type": "number", "minimum": 0},
+          "tokens": {"type": "integer", "minimum": 0},
+          "source": {"type": "string"},
+          "id": {"type": ["string", "null"]},
+          "name": {"type": ["string", "null"]},
+          "kind": {"type": ["string", "null"]},
+          "file": {"type": ["string", "null"]},
+          "line": {"type": ["integer", "null"]},
+          "score": {"type": "number"},
+          "token_cost": {"type": "integer"},
+          "summary": {"type": "string"},
+          "source_pruned": {"type": "boolean"},
+          "source_kept_lines": {"type": "integer"},
+          "source_elided_lines": {"type": "integer"},
+          "source_total_lines": {"type": "integer"},
+          "source_is_pruned_view": {"type": "boolean"}
+        },
+        "additionalProperties": true
+      }
+    },
+    "total_tokens": {"type": "integer", "minimum": 0},
+    "budget_tokens": {"type": "integer", "minimum": 1},
+    "items_included": {"type": "integer", "minimum": 0},
+    "items_considered": {"type": "integer", "minimum": 0},
+    "negative_evidence": {
+      "type": "object",
+      "required": ["verdict", "scanned_symbols", "scanned_files", "best_match_score"],
+      "properties": {
+        "verdict": {"enum": ["no_implementation_found", "low_confidence_matches"]},
+        "scanned_symbols": {"type": "integer", "minimum": 0},
+        "scanned_files": {"type": "integer", "minimum": 0},
+        "best_match_score": {"type": "number", "minimum": 0},
+        "related_existing": {"type": "array", "items": {"type": "string"}}
+      },
+      "additionalProperties": true
+    },
+    "⚠ warning": {"type": "string"},
+    "_meta": {
+      "type": "object",
+      "required": ["timing_ms", "tokens_saved", "total_tokens_saved"],
+      "properties": {
+        "timing_ms": {"type": "number", "minimum": 0},
+        "tokens_saved": {"type": "integer", "minimum": 0},
+        "total_tokens_saved": {"type": "integer", "minimum": 0},
+        "verdict": {"type": "object", "description": "Conforms to retrieval-verdict.schema.json."},
+        "freshness": {"type": "object"},
+        "confidence": {},
+        "query_shape": {
+          "type": "object",
+          "required": ["source_shaped", "exact_seeded"],
+          "properties": {
+            "source_shaped": {"type": "array", "items": {"type": "string"}, "minItems": 1},
+            "exact_seeded": {"type": "integer", "minimum": 0}
+          },
+          "additionalProperties": false
+        }
+      },
+      "additionalProperties": true
+    }
+  },
+  "additionalProperties": true
+}

schemas/retrieval-verdict.schema.json

@@ -0,0 +1,44 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://jcodemunch.com/schemas/retrieval-verdict.schema.json",
+  "title": "jCodeMunch retrieval verdict",
+  "description": "The unified _meta.verdict block attached to retrieval responses (search_symbols, search_text, get_ranked_context). Grounded symbolic retrieval proves absence: an 'absent' state carries the scan counts that back the claim, and 'degraded' means the scan was partial so absence is NOT proven.",
+  "type": "object",
+  "required": ["state", "scanned", "channels", "note"],
+  "properties": {
+    "state": {
+      "enum": ["ok", "low_confidence", "absent", "degraded"],
+      "description": "ok = confident matches; low_confidence = non-empty but below threshold; absent = nothing matched and the scan was complete; degraded = partial scan (timeout, or semantic requested with no embedder) — absence not proven."
+    },
+    "scanned": {
+      "type": "object",
+      "required": ["symbols", "files"],
+      "properties": {
+        "symbols": {"type": "integer", "minimum": 0},
+        "files": {"type": "integer", "minimum": 0}
+      },
+      "additionalProperties": false
+    },
+    "best_score": {
+      "type": ["number", "null"],
+      "description": "Best raw match score backing the state, null when no scoring ran."
+    },
+    "channels": {
+      "type": "object",
+      "required": ["lexical", "semantic", "index"],
+      "properties": {
+        "lexical": {"enum": ["ok"]},
+        "semantic": {"enum": ["ok", "off", "unavailable"]},
+        "index": {"enum": ["fresh", "stale"]}
+      },
+      "additionalProperties": false
+    },
+    "note": {"type": "string"},
+    "did_you_mean": {
+      "type": "array",
+      "items": {"type": "string"},
+      "description": "Near-miss suggestions when the state is absent/low_confidence."
+    }
+  },
+  "additionalProperties": true
+}