fix(calibration): PresenceSpecialist reads empty room as present when occ_var < empty_var (#1440) - #1448
Merged
Merged
Conversation
… occ_var < empty_var (#1440) PresenceSpecialist::train() set its variance decision threshold to the midpoint between empty-room and occupied-anchor variance, assuming occupied windows are always noisier than the empty baseline. A still, quiet occupant can measure LESS variance than an empty room's own ambient/interference noise floor -- in that case the midpoint sits BELOW the empty-room baseline itself, so a genuinely empty room reads "present" on every single frame, at nonzero confidence. This matches a secondary finding from issue #1440's controlled reproduction: after room calibration (train_empty_room/ train_still_presence/train_walking/train_active_movement), a known- empty room read "present" 31/31 frames. The reporter's primary finding (raw-CSI amplitude-only presence sits at chance, AUC ~= 0.576, once time-of-day is controlled) is a genuine sensing-difficulty result, not a code bug, and is already addressed by the maintainer's prior comment pointing to this per-room calibration path -- this fix addresses the part of the report that IS a code bug: the calibration path itself mis-behaving on presence classification, worse than chance (constant wrong answer) rather than merely noisy. Fix: only trust the variance channel when occupied variance exceeds the empty baseline by a margin (VARIANCE_SEPARATION_MARGIN = 5%); otherwise disable it (threshold = +inf), mirroring how the mean-shift channel already goes inert when the anchors' means don't separate. Also fixed a latent confidence-scoring bug this exposed: an infinite threshold previously drove the variance confidence term to a spurious 1.0 via `(x - inf).abs() / span`, which would have overstated confidence in a disabled channel's "vote". Added `presence_inverted_variance_never_reports_empty_room_as_present`, pinning the exact scenario (occ_var < empty_var, identical means, no mean-shift channel to fall back on) that isolates the variance-only bug. All 64 existing + new tests pass, 0 regressions. Fixes #1440 Co-Authored-By: claude-flow <ruv@ruv.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses part of #1440.
Scope — what this fixes and what it doesn't
Issue #1440 is a rigorous controlled reproduction with two distinct findings:
train_empty_room/train_still_presence/train_walking/train_active_movement), a known-empty room read "present" 31/31 frames, and training accuracy was 0.45 on 4 classes. This part is a code bug, and this PR fixes it.Root cause
PresenceSpecialist::train()(v2/crates/wifi-densepose-calibration/src/specialist.rs) sets its variance decision threshold at the midpoint between the empty-room baseline variance and the mean occupied-anchor variance, assuming occupied windows are always noisier than empty ones. A still, quiet occupant (train_still_presence) can measure less variance than an empty room's own ambient/interference noise floor — in that case the midpoint sits below the empty-room baseline itself, so a genuinely empty room reads "present" on every frame, at nonzero confidence. This is consistent with the reporter's 31/31 symptom and a training accuracy barely above chance for 4 classes.Fix
Only trust the variance channel when occupied variance exceeds the empty baseline by a margin (
VARIANCE_SEPARATION_MARGIN = 5%); otherwise disable it (threshold = +inf), mirroring how the mean-shift channel already goes inert when the anchors' means don't separate (existingmean_dist_threshold: Option<f32>pattern). Also fixed a latent confidence-scoring bug this exposed: an infinite threshold previously drove the variance confidence term to a spurious1.0via(x - inf).abs() / span, which would have overstated confidence in a disabled channel's contribution.Testing
presence_inverted_variance_never_reports_empty_room_as_present, isolating the exact scenario (occ_var < empty_var, identical means so there's no mean-shift fallback) that reproduces the bug pre-fix and confirms the fix post-fix.cargo test -p wifi-densepose-calibration --no-default-features: 63 unit + 1 integration test, 0 failed, 0 regressions (all pre-existing presence/mean-shift/backward-compat tests still pass).What's still open
The reporter's deeper questions (is phase the right lever? is
ruview-unifiedthe intended future path? placement guidance?) are legitimate open research questions this fix doesn't answer — flagging so the issue isn't fully closed by this alone, just the concrete regression within it.Co-Authored-By: claude-flow ruv@ruv.net