Skip to content

Commit a4cc927

Browse files
jgravelleclaude
andcommitted
release: v1.108.116 — unified retrieval verdict (_meta.verdict)
Unify the duplicated negative_evidence logic in search_symbols and get_ranked_context into retrieval/verdict.py::build_verdict, complete the taxonomy (ok/low_confidence/absent/degraded), add a new `degraded` state for silently-downgraded channels (semantic requested with no embedder, or a timed-out text scan), and extend _meta.verdict coverage to search_text. Backward-compatible: the legacy negative_evidence dict and warning fire on the unchanged trigger from the same helper; _meta.verdict is additive. No INDEX_VERSION bump, no tool-count or input-schema change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 79903ba commit a4cc927

8 files changed

Lines changed: 328 additions & 43 deletions

File tree

CHANGELOG.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,37 @@
22

33
All notable changes to jcodemunch-mcp are documented here.
44

5+
## [1.108.116] - 2026-07-10 - Unified retrieval verdict (`_meta.verdict`)
6+
7+
### Added
8+
9+
- **A unified `_meta.verdict` on the retrieval tools** — one honesty contract with a
10+
complete taxonomy: `ok` / `low_confidence` / `absent` / `degraded`. A weak or empty
11+
result is positive, token-saving evidence: grounded symbolic retrieval can prove
12+
"this is not here," so the agent stops reformulating a query for something the index
13+
provably does not contain. Each verdict carries the scan counts that back an absence
14+
claim, per-channel status (`lexical` / `semantic` / `index`), and near-miss
15+
`did_you_mean` suggestions.
16+
- **`degraded` is a new state** that fires when a requested retrieval channel was
17+
silently downgraded — semantic search requested with no embedding provider
18+
configured, or a cut-short (`timed_out`) text scan — so a partial result is never
19+
mistaken for a proof of absence.
20+
- **`search_text` now emits `_meta.verdict`** (previously it returned silent empties
21+
with no absent/ok signal).
22+
23+
### Changed
24+
25+
- The previously-duplicated `negative_evidence` computation in `search_symbols` and
26+
`get_ranked_context` is unified into one helper, `retrieval/verdict.py::build_verdict`.
27+
The legacy top-level `negative_evidence` dict and `⚠ warning` string are produced from
28+
the same computation on the **unchanged** trigger (empty result, or best score below
29+
`negative_evidence_threshold`), so existing consumers and the injected agent policy
30+
are unaffected — `_meta.verdict` is purely additive.
31+
32+
NO INDEX_VERSION bump; no tool-count or input-schema change; inline compute, no new
33+
background or network behavior. New `retrieval/verdict.py`; `tests/test_negative_evidence.py`
34+
extended (10 new cases across the four states, `did_you_mean`, and legacy parity).
35+
536
## [1.108.115] - 2026-07-09 - New installs default to the `counter` tool surface
637

738
### Changed

CLAUDE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# jcodemunch-mcp — Project Brief
22

33
## Current State
4+
- **Version:** 1.108.116 — **Unified retrieval verdict (`_meta.verdict`) — clean-room honesty primitive (competitor unnamed in shipped artifacts).** jcm already shipped scattered `negative_evidence` (verdicts `no_implementation_found`/`low_confidence_matches`, threshold 0.5) on `search_symbols` + `get_ranked_context` plus `_meta.freshness`/`confidence`; this unifies them into ONE `build_verdict()` helper (`retrieval/verdict.py`) with a complete taxonomy `ok`/`low_confidence`/`absent`/`degraded`, attached as `_meta.verdict` (scan counts backing an absence claim + per-channel lexical/semantic/index status + `did_you_mean` near-misses). **`degraded` is new** (semantic requested with no embedder configured — reuses `embed_repo._detect_provider` — or a `timed_out` text scan → partial result, absence NOT proven). **`search_text` gains a verdict** (was silent empties). The thesis wedge: grounded symbolic retrieval PROVES "not here" where nearest-neighbour RAG always returns its closest something, so the agent stops re-querying for code the index provably lacks. **Backward-compatible:** the legacy `negative_evidence` dict + `⚠ warning` fire on the unchanged trigger (empty or below-threshold) from the same helper; `_meta.verdict` is additive. Refactored the two duplicated ne blocks (search_symbols ~L1035, get_ranked_context ~L390) to compute after the freshness probe so the verdict can report index staleness. NO INDEX_VERSION bump, no tool-count/input-schema change, inline compute (no bg/network → no README bg-disclosure). New `retrieval/verdict.py`; `tests/test_negative_evidence.py` +10 (four states, did_you_mean, degraded-on-semantic-unavailable via monkeypatch, legacy parity). Origin: PRD `C:\MCPs\business\jcm-retrieval-verdict\PRD.md` (Phase 2 = file-tool did-you-mean; Phase 3 = jdoc/jdata parity). Local-only full-suite note: 12 `test_semantic_search.py` failures appear when a local embed model sits in `~/.code-index/models/` (provider-detection tests expect a clean env) — verified pre-existing/env, NOT this change (they pass with the model absent, as in CI).
45
- **Version:** 1.108.115 — **New installs default to `tool_surface="counter"`; existing installs never collapse on update.** Origin: a competitive-intel read of Comet/Opik's "AI coding cost optimization" post (their MCP went dozens-of-tools → 5 composable ops for a ~100× prompt-overhead cut) prompted a token-cost analysis of jcm's surface. Measured (tiktoken proxy) resident `list_tools` schema cost: **full (93 tools) 21,614 tok / core (20) 5,332 / counter (6) 882** — `standard` (81 tools) is only ~8% under full, a non-lever. Conclusion: the tools are cheap to OWN (folded away by tiering/the Counter); the cost is ADVERTISING them every turn. So the lever is the default surface, NOT retiring "redundant" tools (redundancy audit was answering the wrong question — the ~110 tools survived real audits and each earns its slot when needed; a folded tool costs ~0). jjg's call: hard `counter` default for new users, but **UPDATES must not degrade** existing users. Implemented as a first-ever-install gate: code default stays `"full"`; `_fresh_config_content()` ACTIVATES `counter` only when `_is_first_ever_install()` (no prior `.db`/`.sqlite` in `CODE_INDEX_PATH`) AND the config is being created fresh; the template ships `tool_surface` **commented** under `=== Tool Surface ===` so `upgrade_config` (which back-injects template keys) can only ever add the inactive form. `tool_surface` added to `DEFAULTS`/`CONFIG_TYPES` (was env-only → an explicit config value now honored; `JCODEMUNCH_TOOL_SURFACE` env still wins). New `set_string_key()` mirrors `set_bool_key`. Front door verified self-bootstrapping for a cold model (`order`/`menu`/`route` cross-reference; `route(execute=true)` collapses discover→dispatch to one round-trip), so `counter` is safe for a first-touch user. All 3 config-creation sites (load_config / apply_share_savings / set-key) route through `_fresh_config_content`. NO INDEX_VERSION bump, tool count unchanged. New `tests/test_v1_108_115.py` (16); config/counter/config_set suites green (229). Telemetry note reconfirmed: org-rollup ingest payload is `{org_id,seat_id,tokens_saved,usd,calls,date}` only — NO per-tool data crosses the wire (minimalist by design; not widened).
56
- **Version:** 1.108.114 — **`route()` gains mutate + stateful intent buckets (the leakage finding from the router-triage lab).** The Counter's `route()`/`classify_intent` (`counter.py` `_INTENT_RULES`) had no bucket for edit-commands or session/recent-change queries → they fell through to `search_symbols` or NO_MATCH (measured: mutate+stateful 95% NO_MATCH against a 140-request corpus). Added both, on-charter: **stateful** ("what did I just change", "uncommitted edits", "different from main", "recap this session") → `get_changed_symbols` / `get_session_context`; placed FIRST because stateful phrasings carry trigger words for the impact/reference rules ("affected by my recent changes"). **mutate** ("rename X", "delete the dead code", "refactor the auth module", "add a retry") → recognize-and-redirect to the READ-ONLY prep tool for that edit kind (`check_rename_safe`/`check_delete_safe`/`plan_refactoring`/`check_edit_safe`) — jcm performs no edit, the agent applies with its own editor. **Anchored to a LEADING imperative verb** so a QUESTION about an edit ("what breaks if I rename X") falls through to the impact rules, not the mutate bucket. Recognize-only: no `_QUERY_ARG` entry for the prep tools, so `route(execute=true)` never auto-runs a mutation flow. Verified 20/20 mutate + 20/20 stateful on the corpus, zero regression to existing routes. Charter guard test asserts every mutate recommendation is non-state-changing + doesn't trip the exec/write tripwire. New `tests/test_v1_108_114.py` (9). NO INDEX_VERSION bump; pure routing logic, no schema/tool-count change. Origin: `C:\MCPs\request-router-lab\` triage/cross-check (memory `project_tool_router_prd`).
67
- **Version:** 1.108.113 — **New tool `get_architecture_metrics` (F-15 #4 from the tokensave review, clean-room; competitor unnamed in shipped artifacts). tokensave F-15 series COMPLETE.** Structural concentration + dependency depth + modularity in ONE read-only tool (vs their 3 separate gini/dependency_depth/dsm tools — tool-surface discipline). Over the file import graph (`_build_adjacency` reused from get_dependency_graph): **concentration** = `_gini()` coefficient (0 even→1 hoarded, standard sorted-rank formula) over per-file symbol-count/bytes/fan_in/fan_out + top concentrators (answers "is coupling HOARDED?" which a hotspot peak-list can't); **depth** = Lakos levelization (longest dependency chain + level histogram) over the SCC-condensed DAG via Kahn topo, `_find_cycles` reused for condensation; **modularity** = WCC cluster count + `back_edge_count` (cycle-participating import edges = the hidden coupling a DSM highlights) — NO N×N matrix (that's the bloat we avoid; layering-violation detail stays in get_layer_violations, cycles in get_dependency_cycles — deliberately NOT duplicated). **Does NOT touch the health-radar composite** (observatory comparability preserved — the reason it's a separate tool not new radar axes). Degrades honestly to symbols/bytes Gini when no import data. Read-only analytics, no savings _meta. New `tools/get_architecture_metrics.py`; Architecture group, standard tier (**core_compact unchanged 3969**), tool count **90** full, no CLI, no INDEX_VERSION bump. New `tests/test_architecture_metrics.py` (10). **All 4 tokensave F-15s now done: #1 token-economy=done-by-existing, #2 parity map v1.108.111, #3 decorator census v1.108.112, #4 architecture metrics v1.108.113.**

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "jcodemunch-mcp"
3-
version = "1.108.115"
3+
version = "1.108.116"
44
description = "Token-efficient MCP server for source code exploration via tree-sitter AST parsing"
55
readme = "README.md"
66
requires-python = ">=3.10"
Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
"""Unified retrieval verdict — one honesty contract across the search tools.
2+
3+
An empty or weak retrieval result is positive, token-saving evidence: grounded
4+
symbolic retrieval can prove "this is not here" where nearest-neighbour search
5+
always returns its closest something. ``build_verdict`` centralises the logic
6+
that ``search_symbols`` and ``get_ranked_context`` previously duplicated, and
7+
extends it to ``search_text``.
8+
9+
The result carries two things:
10+
11+
* ``verdict`` — the unified ``_meta.verdict`` dict with a complete taxonomy
12+
(``ok`` / ``low_confidence`` / ``absent`` / ``degraded``), the scan counts that
13+
back an absence claim, per-channel status, and near-miss suggestions.
14+
* ``negative_evidence`` — the legacy dict (or ``None``) with the same trigger and
15+
shape as before, so existing consumers and the injected agent policy keep
16+
working unchanged.
17+
"""
18+
19+
from __future__ import annotations
20+
21+
from typing import Optional, Sequence
22+
23+
# Emitted as verdict["state"].
24+
STATE_OK = "ok"
25+
STATE_LOW_CONFIDENCE = "low_confidence"
26+
STATE_ABSENT = "absent"
27+
STATE_DEGRADED = "degraded"
28+
29+
_NOTES = {
30+
STATE_OK: "Confident matches returned.",
31+
STATE_LOW_CONFIDENCE: (
32+
"Matches are below the confidence threshold; verify before relying on them."
33+
),
34+
STATE_ABSENT: (
35+
"No match found after scanning the index. Treat this as strong evidence the "
36+
"target is not present; do not reformulate the same query expecting a hit."
37+
),
38+
STATE_DEGRADED: (
39+
"A requested retrieval channel was unavailable or the scan was cut short. "
40+
"Results are partial and absence is NOT proven."
41+
),
42+
}
43+
44+
45+
def _semantic_provider_available() -> bool:
46+
"""Return True when an embedding provider is actually configured.
47+
48+
Reuses ``embed_repo``'s live detection so we do not drift from the encoder the
49+
semantic path would really use. Called only when semantic was requested.
50+
"""
51+
try:
52+
from ..tools.embed_repo import _detect_provider
53+
54+
detected = _detect_provider()
55+
if isinstance(detected, tuple):
56+
return bool(detected and detected[0])
57+
return bool(detected)
58+
except Exception:
59+
return False
60+
61+
62+
def _did_you_mean(
63+
source_files: Optional[Sequence[str]],
64+
query_terms: Optional[Sequence[str]],
65+
cap: int = 5,
66+
) -> list:
67+
"""Files whose basename contains a query term (near-miss candidates)."""
68+
if not source_files or not query_terms:
69+
return []
70+
out: list = []
71+
seen: set = set()
72+
for f in source_files:
73+
base = f.lower().replace("\\", "/").rsplit("/", 1)[-1]
74+
if any(t in base for t in query_terms):
75+
if f not in seen:
76+
seen.add(f)
77+
out.append(f)
78+
if len(out) >= cap:
79+
break
80+
return out
81+
82+
83+
def build_verdict(
84+
*,
85+
result_count: int,
86+
scanned_symbols: int = 0,
87+
scanned_files: int = 0,
88+
best_score: Optional[float] = None,
89+
threshold: Optional[float] = None,
90+
query_terms: Optional[Sequence[str]] = None,
91+
source_files: Optional[Sequence[str]] = None,
92+
semantic_requested: bool = False,
93+
index_stale: bool = False,
94+
timed_out: bool = False,
95+
) -> dict:
96+
"""Compute the unified verdict plus the legacy negative_evidence dict.
97+
98+
Returns ``{"verdict": <_meta.verdict>, "negative_evidence": <dict|None>}``.
99+
100+
Backward compatibility: ``negative_evidence`` fires on exactly the historical
101+
trigger (empty result, or best score below threshold) with the historical keys
102+
and verdict names, so existing tests and the agent policy are unaffected. The
103+
new ``verdict`` is purely additive.
104+
"""
105+
terms = [t for t in (query_terms or []) if t]
106+
did_you_mean = _did_you_mean(source_files, terms)
107+
108+
semantic_available = _semantic_provider_available() if semantic_requested else True
109+
below_threshold = (
110+
threshold is not None and best_score is not None and best_score < threshold
111+
)
112+
113+
# --- unified state (degraded takes precedence: partial scans can't prove absence) ---
114+
if timed_out:
115+
state = STATE_DEGRADED
116+
elif semantic_requested and not semantic_available:
117+
state = STATE_DEGRADED
118+
elif result_count == 0:
119+
state = STATE_ABSENT
120+
elif below_threshold:
121+
state = STATE_LOW_CONFIDENCE
122+
else:
123+
state = STATE_OK
124+
125+
if semantic_requested and not semantic_available:
126+
semantic_channel = "unavailable"
127+
elif semantic_requested:
128+
semantic_channel = "ok"
129+
else:
130+
semantic_channel = "off"
131+
132+
verdict = {
133+
"state": state,
134+
"scanned": {"symbols": int(scanned_symbols), "files": int(scanned_files)},
135+
"best_score": round(best_score, 3) if best_score is not None else None,
136+
"channels": {
137+
"lexical": "ok",
138+
"semantic": semantic_channel,
139+
"index": "stale" if index_stale else "fresh",
140+
},
141+
"note": _NOTES[state],
142+
}
143+
if did_you_mean:
144+
verdict["did_you_mean"] = did_you_mean
145+
146+
# --- legacy negative_evidence: unchanged trigger + shape ---
147+
negative_evidence = None
148+
if result_count == 0 or below_threshold:
149+
negative_evidence = {
150+
"verdict": (
151+
"no_implementation_found" if result_count == 0 else "low_confidence_matches"
152+
),
153+
"scanned_symbols": int(scanned_symbols),
154+
"scanned_files": int(scanned_files),
155+
"best_match_score": round(best_score, 3) if best_score else 0.0,
156+
}
157+
if did_you_mean:
158+
negative_evidence["related_existing"] = did_you_mean
159+
160+
return {"verdict": verdict, "negative_evidence": negative_evidence}

src/jcodemunch_mcp/tools/get_ranked_context.py

Lines changed: 16 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -320,30 +320,15 @@ def _get_tokens(sym):
320320
**_compact_fields(sym, adj_score, item_tokens),
321321
})
322322

323-
# Negative evidence for low-confidence or empty results
323+
# Retrieval verdict threshold. The verdict itself is computed after the
324+
# freshness probe below so it can report index staleness.
324325
_ne_threshold = _NEGATIVE_EVIDENCE_THRESHOLD
325326
try:
326327
from .. import config as _cfg
327328
_ne_threshold = _cfg.get("negative_evidence_threshold", _NEGATIVE_EVIDENCE_THRESHOLD)
328329
except Exception:
329330
pass
330-
331331
negative_evidence = None
332-
if not context_items or max_bm25 < _ne_threshold:
333-
verdict = "no_implementation_found" if not context_items else "low_confidence_matches"
334-
related_existing = [
335-
f for f in index.source_files
336-
if any(t in f.lower().rsplit("/", 1)[-1].rsplit("\\", 1)[-1]
337-
for t in query_terms)
338-
][:5]
339-
negative_evidence = {
340-
"verdict": verdict,
341-
"scanned_symbols": items_considered,
342-
"scanned_files": len(set(s.get("file", "") for _, _, _, s in scored)),
343-
"best_match_score": round(max_bm25, 3),
344-
}
345-
if related_existing:
346-
negative_evidence["related_existing"] = related_existing
347332

348333
# Token savings estimate
349334
raw_bytes = sum(
@@ -381,6 +366,20 @@ def _get_tokens(sym):
381366
_probe.annotate(context_items)
382367
result["_meta"]["freshness"] = _probe.summary(context_items)
383368
_attach_confidence(result, context_items, is_stale=_probe.repo_is_stale)
369+
from ..retrieval.verdict import build_verdict as _build_verdict
370+
_vres = _build_verdict(
371+
result_count=len(context_items),
372+
scanned_symbols=items_considered,
373+
scanned_files=len(set(s.get("file", "") for _, _, _, s in scored)),
374+
best_score=max_bm25,
375+
threshold=_ne_threshold,
376+
query_terms=query_terms,
377+
source_files=index.source_files,
378+
semantic_requested=False,
379+
index_stale=_probe.repo_is_stale,
380+
)
381+
negative_evidence = _vres["negative_evidence"]
382+
result["_meta"]["verdict"] = _vres["verdict"]
384383
_record_ranking_event(
385384
tool="get_ranked_context",
386385
repo=f"{owner}/{name}",

0 commit comments

Comments
 (0)