fix(listening): the shared-reference guard closed a quarter of its defect

Third audit round, aimed at the second round's own fixes. Two findings, the
second of which explains how the first got shipped.

1. EXACT-MATCH MERGING ONLY CATCHES A RE-USED FILE.
capture_audio records live playback, so a producer who re-captures their
reference each session gets a NEW file every time -- same material, never
byte-identical. Measured at production geometry, 6 sessions x 2 pairs:

  re-recorded anchor cosine   0.98  0.95  0.90  0.85  0.80  0.70
  certified as real taste     100%  100%  100%  100%  100%  100%
  captures merged                0     0     0     0     0     0
  (honest control, own reference per session: 1%)

A cosine threshold cannot fix this: at anchor cosine 0.70 the two anchors
are less alike than two independent captures from one project, so any
cutoff strict enough to catch it merges every real corpus.

Added _anchor_asymmetry -- mean cross-session cosine among rejected
captures minus the same among preferred. Relative, so it needs no
calibration constant:

  anchored 0.70 .. 1.00            +0.47 .. +1.00   (sd ~0.006)
  honest, own reference/session    -0.000           (sd  0.008)
  honest + a real quality signal   +0.002           (sd  0.011)
  reference on the PREFERRED side  -0.810           (sd  0.005)

A genuine preference moves both sides apart symmetrically, which is why it
does not trip. After: 0% certified at every cosine, control untouched, and
a real signal still certifies at 100%.

2. THE MEASUREMENTS WERE TAKEN IN A REGIME THE CODE NEVER RUNS IN.
Real CLAP embeddings are L2-normalised, so difference vectors have norm
0.3-1.4. Every number quoted for the earlier guards was measured on raw
gaussians at norm ~30, where _fit_weights returns ~1e-8 and every corpus
scores at chance. That is how a guard closing a quarter of its defect
passed a full green suite with a docstring claiming it was measured.

Test helpers now generate unit-norm captures at production dimension, and
every documented number was re-measured. The session-level false-positive
residual is 5.8-12.5% (previously documented ~9%), degrading as sessions
get thinner. Power unchanged: 100% at >=5 sessions, 0% at 4.

Also documents _fit_weights for what it is: at _LR=0.5 and l2=1.0 the
memory coefficient (1 - _LR*2*l2) is exactly 0, so it is a fixed-point
iteration, not gradient descent. Correct in the deployed regime, fragile to
any change in those two constants.

Corrects the previous commit's claim of "After the fix: 0 of 40", which
held only for byte-identical captures.

Suite 4784 passed, 2 skipped. Tool count unchanged at 472.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: dreamrec

CHANGELOG.md

@@ -67,7 +67,8 @@ looking. Both defects are measured, not argued.
   of 40 runs, for a head scoring 50.7% on fresh pairs. The identical corpus without a
   shared anchor: 3 of 40. Groups sharing a capture are now merged before scoring, which
   routes them to the existing OPTIMISTIC path, withholds the p-value and makes
-  `taste_rank` warn. After the fix: 0 of 40. Reported as `groups_merged`.
+  `taste_rank` warn. Reported as `groups_merged`. **Superseded below — that fix closed
+  only the re-USED-file case, roughly a quarter of the defect.**
 - **Significance counted pairs as independent trials.** Grouping the CV split left the
   test itself ungrouped, but every pair in a held-out session is scored by one fitted `w`
   against correlated material, so a session is one observation rather than k. False
@@ -86,7 +87,41 @@ looking. Both defects are measured, not argued.
   regardless of corpus size). A test that cannot certify anything is not conservative, it
   is broken — and it would have looked more rigorous than what it replaced. The reasoning
   is recorded in `_group_sign_test` so it is not re-attempted blind.
-- 5 new regression tests pin the two defects; the suite is 4780 passed / 2 skipped.
+- 5 new regression tests pin the two defects.
+
+### Fixed — the shared-reference guard closed a quarter of its defect, and the measurements said otherwise
+A third audit round, aimed at the previous round's own fixes.
+
+- **Exact-match merging only catches a re-USED file.** `capture_audio` records live
+  playback, so a producer who re-captures their reference each session produces a new
+  file every time — same material, never byte-identical. Measured at production geometry,
+  6 sessions x 2 pairs: a re-recorded anchor at cosine 0.98 / 0.95 / 0.90 / 0.85 / 0.80 /
+  0.70 certified as real taste in **100%** of runs at every one, merging **nothing**,
+  against 1% for a control with an independent reference per session. A cosine threshold
+  cannot fix it: at 0.70 the two anchors are less alike than two independent captures from
+  one project, so any cutoff strict enough would merge every real corpus.
+- Added `_anchor_asymmetry` — mean cross-session cosine among rejected captures minus the
+  same among preferred. It needs no calibration constant because it is relative: anchored
+  corpora measure **+0.47 to +1.00** (sd ~0.006), honest ones **-0.000** (sd 0.008), and a
+  corpus carrying a genuine quality signal **+0.002** (sd 0.011) — a real preference moves
+  both sides apart symmetrically, so it does not trip. Symmetric in which side is anchored
+  (a shared preferred capture measures -0.81). After: 0% certified at every cosine, control
+  untouched, genuine signal still certifies at 100%. Reported as `anchor_asymmetry` and
+  `shared_reference_detected`.
+- **The measurements themselves were the deeper defect.** Real CLAP embeddings are
+  L2-normalised, so difference vectors have norm 0.3-1.4. Every number quoted for the
+  earlier guards was measured on raw gaussians at norm ~30, where `_fit_weights` returns
+  ~1e-8 and every corpus scores at chance. That regime is why a guard closing a quarter of
+  its defect passed a full green suite with a docstring claiming it was measured. Test
+  helpers now generate unit-norm captures at production dimension, and every documented
+  number was re-measured: the session-level false-positive residual is **5.8-12.5%**
+  (previously documented as ~9%), degrading as sessions get thinner; power is unchanged at
+  100% for >=5 sessions.
+- `_fit_weights` is documented for what it is. At `_LR=0.5` and `l2=1.0` the memory
+  coefficient `1 - _LR*2*l2` is exactly 0, so it is a fixed-point iteration rather than
+  gradient descent. Correct in the deployed regime, fragile to any change in those two
+  constants.
+- 4 new regression tests; the suite is 4784 passed / 2 skipped.
 
 ## v1.28.0 — 2026-07-31
 

livepilot/skills/livepilot-core/references/perception.md

@@ -87,11 +87,15 @@ itself — every item below was added after it was caught doing exactly that:
 - Significance counts **sessions, not pairs** — five or more separate sessions,
   or nothing can be certified however good the accuracy looks. Counting pairs
   put the false-positive rate at ~25% against a nominal 5%.
-- **Do not A/B every candidate against one fixed baseline.** Pairs sharing a
-  capture are one piece of evidence, not several; they are merged before
-  scoring, and `groups_merged` in the report says when that happened. Before
-  this guard existed, 6 such pairs across 3 labelled sessions reported "a real
-  preference signal" in 40 of 40 runs — for a head scoring 50.7% on fresh pairs.
+- **Do not A/B every candidate against one fixed baseline** — not even a
+  re-captured one. Pairs sharing a reference are one piece of evidence, not
+  several. Two guards catch it: `groups_merged` for a reused file,
+  `shared_reference_detected` / `anchor_asymmetry` for a re-recorded one (which
+  is what `capture_audio` produces — it records live playback, so the file
+  differs every time even when the material does not). Before these, a
+  re-recorded reference certified as real taste in **100%** of runs at every
+  anchor similarity from 0.70 to 0.98, against 1% for an honest control.
+  Compare candidates against **each other**.
 
 `significant: null` means "could not be tested" (too few sessions, or captures
 shared). Treat it exactly as `false`.

mcp_server/listening/taste_head.py

@@ -28,7 +28,7 @@
 read ~100% whether or not the head learned anything real.
 
 So this module never reports training accuracy. It reports cross-validated
-accuracy, a p-value against chance, and a verdict in words. Four corrections
+accuracy, a p-value against chance, and a verdict in words. Five corrections
 make that honest, each added after it was caught misreporting:
 
 1. GROUPED cross-validation. A session yields several pairs from the same
@@ -43,21 +43,38 @@
    learnable signal at the same n. `significant: false` means the number
    carries no information however high it looks.
 
-3. SHARED-CAPTURE merging, before the split. Grouping trusts a label, and no
-   label can say "these pairs share a baseline". Keep one reference render and
-   A/B candidates against it — the most natural way anyone compares anything —
-   and every pair carries the same anchor, so every fold learns "not the
-   anchor" and scores perfectly. Measured: 6 pairs over 3 labelled sessions
-   sharing one anchor reported 100% / p=0.016 / significant in 40 of 40 runs,
-   for a head that scores 0.507 on fresh pairs. See
-   `_merge_groups_sharing_a_capture`.
+3. SHARED-REFERENCE detection, before the split. Grouping trusts a label, and
+   no label can say "these pairs share a baseline". Keep one reference render
+   and A/B candidates against it — the most natural way anyone compares
+   anything — and every pair carries the same anchor, so every fold learns "not
+   the anchor" and scores perfectly. Two guards, because the obvious one closes
+   only a quarter of it: `_merge_groups_sharing_a_capture` catches a re-USED
+   file, and `_anchor_asymmetry` catches a re-RECORDED one, which is what
+   `capture_audio` actually produces. Measured at production geometry, 6
+   sessions x 2 pairs, before the second guard existed:
+
+       re-recorded anchor cosine   0.98  0.95  0.90  0.85  0.80  0.70
+       certified as real taste     100%  100%  100%  100%  100%  100%
+       captures merged                0     0     0     0     0     0
+
+   (honest control, own reference per session: 1%.) After: 0% certified at
+   every cosine, control untouched, and a genuine quality signal still
+   certifies at 100%.
 
 4. Significance over SESSIONS. Correction 1 grouped the cross-validation split
    but left the test counting pairs as independent trials. False-positive rate
    on corpora with no transferable signal, nominal 5%: 25% counting pairs, ~9%
    counting sessions. See `_group_sign_test`, which also records why the
    textbook permutation fix was built and rejected.
 
+5. Measuring any of this in the geometry the code ACTUALLY RUNS IN. Real CLAP
+   embeddings are L2-normalised, so a difference vector has norm 0.3-1.4. The
+   first two rounds of measurement used raw gaussians at norm ~30, where
+   `_fit_weights` returns near-zero and every corpus looks clean. That is how a
+   guard closing a quarter of its defect passed a full green suite with a
+   docstring claiming it was measured. The test helpers now generate unit-norm
+   captures at production dimension; see `_unit` in the test module.
+
 The cost of 3 and 4 is a higher bar: significance needs decisions from
 MIN_GROUPS_FOR_SIGNIFICANCE separate sessions with no shared captures, and below
 that nothing can be certified however good it looks. That bar is derived from
@@ -153,8 +170,21 @@ def _update(data: dict) -> dict:
 def _fit_weights(diffs: np.ndarray, l2: float) -> np.ndarray:
     """Logistic regression on difference vectors, all labels positive.
 
-    Minimises  -sum(log sigmoid(w.d)) + l2*||w||^2  by gradient descent.
-    Convex, so the optimum is unique and plain GD is sufficient at this size.
+    Targets  -sum(log sigmoid(w.d)) + l2*||w||^2, which is convex.
+
+    Note what the iteration actually is at the default settings. The update
+    w -= _LR * (-mean(d*p_wrong) + 2*l2*w) has memory coefficient
+    (1 - _LR*2*l2), which is EXACTLY 0 at _LR=0.5 and l2=1.0 — so w is
+    discarded and recomputed each step rather than accumulated. It is a fixed
+    point iteration, not gradient descent, and it converges in one step for
+    well-scaled inputs.
+
+    That is fine in the deployed regime (unit-norm CLAP embeddings, ||d|| in
+    0.3-1.4, giving ||w|| ~ 0.03-0.26 and clean margins) but it is fragile: at
+    ||d|| ~ 30 the same code returns ~1e-8 and every corpus scores at chance.
+    Anything that changes _LR or DEFAULT_L2 changes the character of this
+    function, not just its speed. Measure in production geometry — see
+    correction 5 in the module docstring for what happens when you do not.
     """
     w = np.zeros(diffs.shape[1], dtype=np.float64)
     n = len(diffs)
@@ -281,8 +311,12 @@ def _group_sign_test(per_group: list[tuple[int, int]]) -> Optional[float]:
     A session counts once, as a win or a loss depending on whether the head beat
     chance on it. Ties carry no directional evidence and are dropped.
 
-    KNOWN RESIDUAL, measured not assumed: ~9% against a nominal 5%, because the
-    CV folds share training data so sessions are not fully independent either.
+    KNOWN RESIDUAL, re-measured at production geometry (unit-norm CLAP, 512-d):
+    5.8% at 6 sessions x 4 pairs, 6.7% at 5 x 4, and 12.5% at 8 x 3 — against a
+    nominal 5%. It degrades as sessions get thinner, because the CV folds share
+    training data so sessions are not fully independent either. Power is
+    unaffected: 100% on a genuine shared direction at >= 5 sessions, 0% at 4
+    (structurally impossible, not a failure).
     The textbook fix — a group sign-flip permutation test — was built and
     REJECTED: `_fit_weights` is deliberately robust (`p_wrong` downweights
     already-correct points), so on the bimodal +/-d data that sign-flipping
@@ -304,13 +338,23 @@ def _group_sign_test(per_group: list[tuple[int, int]]) -> Optional[float]:
 
 def _verdict(loo: Optional[float], n_pairs: int, scheme: str = "",
              n_groups: int = 0, p: Optional[float] = None,
-             n_decisive: int = 0, merged: int = 0) -> str:
+             n_decisive: int = 0, merged: int = 0, anchored: bool = False,
+             asymmetry: float = 0.0) -> str:
     if loo is None:
         return (f"Only {n_pairs} pair(s). Need at least {MIN_PAIRS} before "
                 "cross-validation says anything; the head is not usable yet.")
 
     prefix = ""
-    if merged:
+    if anchored:
+        side = "rejected" if asymmetry > 0 else "preferred"
+        prefix = (f"Every session's {side} captures are far more alike than "
+                  f"chance (asymmetry {asymmetry:+.2f} against ~0.00 for "
+                  "independent material), which is the signature of one shared "
+                  "reference compared against over and over. Those decisions are "
+                  "one piece of evidence, not several, so they were scored as a "
+                  "single session. Compare candidates against EACH OTHER, not "
+                  "against a fixed baseline. ")
+    elif merged:
         prefix = (f"{merged + 1} of the recorded groups share a capture — a "
                   "reused baseline makes those pairs one piece of evidence, not "
                   "several — so they were merged before scoring. ")
@@ -411,6 +455,59 @@ def find(x: str) -> str:
     return [find(g) for g in groups], merged
 
 
+# Honest corpora measure 0.000 +/- 0.008 (max 0.030 over 60 runs, including
+# corpora carrying a genuine quality signal). The weakest anchoring that still
+# certified at 100% measures 0.473. This sits >13 sd above honest and far below
+# the weakest real offender — a gap wide enough that the exact value is not
+# load-bearing.
+ANCHOR_ASYMMETRY_LIMIT = 0.15
+
+
+def _anchor_asymmetry(pairs: list[dict], groups: list[str]) -> float:
+    """How much more alike the REJECTED captures are than the preferred ones,
+    measured only ACROSS sessions, using the groups AS RECORDED. Positive means a shared reference on the
+    rejected side; negative means one on the preferred side.
+
+    Guard 4 keyed on exact vector equality, which turned out to close a far
+    narrower hole than it claimed. `capture_audio` records live playback, so a
+    producer who re-captures their reference each session produces a NEW file
+    every time — never byte-identical, always the same material. Measured at
+    realistic CLAP geometry, 6 sessions x 2 pairs, no genuine quality signal:
+
+        re-recorded anchor cosine   0.98  0.95  0.90  0.85  0.80  0.70
+        certified as real taste     100%  100%  100%  100%  100%  100%
+        captures merged by guard 4     0     0     0     0     0     0
+        (honest control, own anchor per session: 1%)
+
+    A cosine threshold cannot fix that: at anchor cosine 0.70 the two anchors are
+    less alike (0.49) than two independent captures from one project, so any
+    cutoff strict enough to catch it merges every real corpus. This statistic is
+    relative instead, so it needs no notion of "how similar is one project":
+
+        anchored 0.70 .. 1.00            +0.473 .. +1.000  (sd ~0.006)
+        honest, own anchor per session   -0.000            (sd  0.008)
+        honest + a real quality signal   +0.002            (sd  0.011)
+        anchor on the PREFERRED side     -0.810            (sd  0.005)
+
+    A real preference direction moves preferred and rejected captures APART
+    symmetrically, leaving this near zero — which is why a genuine signal does
+    not trip it.
+    """
+    def mean_cross(key: str) -> float:
+        vs = np.array([p[key] for p in pairs], dtype=np.float64)
+        norms = np.linalg.norm(vs, axis=1, keepdims=True)
+        norms[norms == 0] = 1.0
+        vs = vs / norms
+        sims = []
+        for i in range(len(vs)):
+            for j in range(i + 1, len(vs)):
+                if groups[i] != groups[j]:
+                    sims.append(float(vs[i] @ vs[j]))
+        return float(np.mean(sims)) if sims else 0.0
+
+    return mean_cross("rejected") - mean_cross("preferred")
+
+
 def train(store: TasteHeadStore, l2: float = DEFAULT_L2) -> dict:
     """Fit the head and report cross-validated skill (never training fit)."""
     data = store.get_all()
@@ -428,6 +525,15 @@ def train(store: TasteHeadStore, l2: float = DEFAULT_L2) -> dict:
     # Guard #4 runs BEFORE the split: a label cannot express "these pairs share
     # a baseline capture", and grouped CV is blind to it.
     groups, merged = _merge_groups_sharing_a_capture(pairs, raw_groups)
+    # ...and exact equality only catches a re-USED file, not a re-RECORDED one,
+    # which is what capture_audio actually produces. This catches the rest.
+    # Measured against the groups as RECORDED, not the merged ones: the merge
+    # can already have collapsed everything into a single group, leaving no
+    # cross-group pairs and a statistic of 0.0 that would read as "clean".
+    asymmetry = _anchor_asymmetry(pairs, raw_groups)
+    anchored = abs(asymmetry) > ANCHOR_ASYMMETRY_LIMIT
+    if anchored:
+        groups = ["_shared_reference"] * n
     n_groups = len(set(groups))
     weights = _fit_weights(diffs, l2)
     loo, scheme, per_group = _cv_accuracy(diffs, groups, l2)
@@ -454,14 +560,16 @@ def train(store: TasteHeadStore, l2: float = DEFAULT_L2) -> dict:
         "n_groups": n_groups,
         "n_groups_recorded": len({g for g in raw_groups if g}),
         "groups_merged": merged,
+        "anchor_asymmetry": round(asymmetry, 4),
+        "shared_reference_detected": bool(anchored),
         "n_decisive_groups": n_decisive,
         "sessions_needed_for_significance": MIN_GROUPS_FOR_SIGNIFICANCE,
         "baseline_accuracy": 0.5,
         "p_value": None if not certifiable else round(p_value, 4),
         "significant": (None if not certifiable
                         else bool(p_value <= SIGNIFICANCE_ALPHA)),
         "verdict": _verdict(loo, n, scheme, n_groups, p_value, n_decisive,
-                            merged),
+                            merged, anchored, asymmetry),
         "trained_at": int(time.time()),
     }
     # Training accuracy is deliberately absent: with n << d it is ~100%

mcp_server/listening/taste_tools.py

@@ -116,12 +116,15 @@ def taste_record_pair(
     reflect the true grouping.
 
     Two things about HOW you compare, which matter more than how many pairs you
-    record. Do not A/B every candidate against one fixed baseline: pairs that
-    share a capture are one piece of evidence, not several, and they are merged
-    before scoring (the head cannot be certified on them at all). And record
-    from at least ``taste_head.MIN_GROUPS_FOR_SIGNIFICANCE`` separate sessions —
-    significance counts sessions, not pairs, so more pairs from the sessions you
-    already have will not move it.
+    record. Do not A/B every candidate against one fixed baseline — not even a
+    re-captured one, since ``capture_audio`` records live playback and produces
+    a different file each time for the same material. Pairs sharing a reference
+    are one piece of evidence, not several, and a corpus built that way cannot
+    be certified at all (see ``anchor_asymmetry`` in the train report). Compare
+    candidates against EACH OTHER. And record from at least
+    ``taste_head.MIN_GROUPS_FOR_SIGNIFICANCE`` separate sessions — significance
+    counts sessions, not pairs, so more pairs from the sessions you already have
+    will not move it.
 
     Recording invalidates any previously fitted head — rerun ``taste_train``.
     """
@@ -180,9 +183,11 @@ def taste_train(l2: float = taste_head.DEFAULT_L2) -> dict[str, Any]:
     turned out to share captures. Treat ``null`` exactly as you would
     ``false``: as no evidence.
 
-    ``groups_merged`` is worth reading. It is non-zero when sessions you
-    recorded separately shared a capture and were collapsed, which is the
-    single most common way a corpus looks bigger than the evidence in it.
+    ``groups_merged`` and ``shared_reference_detected`` are worth reading. They
+    fire when sessions you recorded separately turn out to share a reference —
+    byte-identical for the first, merely the same material for the second —
+    which is the single most common way a corpus looks bigger than the evidence
+    in it.
 
     ``l2`` raises or lowers regularisation. The default is strong on purpose —
     with far more dimensions than pairs, the penalty is what stops the head