Skip to content

release: v1.108.185 — fusion gets a verdict, and fusion can prove abs… #137

release: v1.108.185 — fusion gets a verdict, and fusion can prove abs…

release: v1.108.185 — fusion gets a verdict, and fusion can prove abs… #137

Workflow file for this run

name: Replay
# Retrieval-quality regression gate. Indexes this repo, replays a fixed set of
# golden queries through search_symbols, and fails if nDCG / MRR / Recall@k drop
# more than 2% (relative) below the committed golden baseline. This is the gate
# that lets ranking-affecting changes (fusion weights, BM25 normalization, parser
# extraction) land with a proof they did not regress retrieval quality.
on:
push:
branches: [main]
pull_request:
permissions:
contents: read
jobs:
replay:
name: Retrieval-quality gate
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
- name: Set up uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7
with:
python-version: "3.12"
- name: Install
run: uv sync --group dev
- name: Index self + run replay gate
run: |
set -euo pipefail
STORE="${RUNNER_TEMP}/replay-index"
# The repo id is a hash of the absolute index path, which differs on a
# CI runner, so capture the actual id and pass it through --repo.
REPO_ID=$(uv run python -c "from jcodemunch_mcp.tools.index_folder import index_folder; print(index_folder(path='.', use_ai_summaries=False, incremental=False, storage_path='${STORE}')['repo'])")
echo "indexed repo: ${REPO_ID}"
uv run python benchmarks/replay/run_replay.py \
--fixture benchmarks/replay/fixtures/self_v1_75_0.json \
--repo "${REPO_ID}" \
--storage-path "${STORE}" \
--baseline-file benchmarks/replay/results/self_v1_75_0-golden.json \
--gate 0.02