Skip to content

fix(chroma): flag stale HNSW ghost surplus - #2262

Draft
fallenmi wants to merge 1 commit into
MemPalace:developfrom
fallenmi:codex/fix-hnsw-ghost-surplus
Draft

fix(chroma): flag stale HNSW ghost surplus#2262
fallenmi wants to merge 1 commit into
MemPalace:developfrom
fallenmi:codex/fix-hnsw-ghost-surplus

Conversation

@fallenmi

Copy link
Copy Markdown

What does this PR do?

Refs #2176.

After repeated or bulk deletes, Chroma can retain substantially more persisted HNSW elements than live SQLite embeddings. The reported case had 1,606 SQLite rows and 9,974 HNSW elements (divergence=-8,368), but the capacity probe classified every negative divergence as OK and explicitly called the surplus safe.

This PR fixes the diagnostic and recovery path:

  • large HNSW surpluses now return status="stale" and repair_recommended=true;
  • the existing diverged flag remains false for this direction, so the native-crash fence reserved for an undersized HNSW index does not disable vector reads or writes;
  • surplus must strictly exceed the same configured threshold used by the capacity probe; small persisted delete lag remains OK;
  • repair-status prints STALE and recommends rebuilding from SQLite, including when all live drawer rows were deleted but ghost HNSW elements remain;
  • MCP emits one warning per distinct stale verdict instead of reporting the state as within tolerance or spamming every refresh;
  • positive SQLite-over-HNSW divergence now also exposes the additive repair_recommended field.

Scope

This is deliberately a detection and recovery-guidance fix. It does not compact or prevent Chroma's deleted HNSW elements, and it does not add cumulative-delete accounting or a hint to every delete_by_source response. Those can be handled separately without coupling a potentially expensive repair to the write path.

Safety

diverged=true currently activates the #1222 fail-closed path: BM25-only search, vector-write refusal, and several preflight aborts. The surplus reported in #2176 degraded ranking but was not shown to create the undersized-index native-crash risk, so the new stale state recommends repair while intentionally leaving vectors enabled.

How to test

python -m pytest -q \
  tests/test_hnsw_capacity.py \
  tests/test_repair.py \
  tests/test_mcp_server.py

pre-commit run --all-files

Local results on current develop:

  • 552 targeted consumer tests passed;
  • 4,301 repository tests passed, 31 skipped, with only the unchanged environment-sensitive test_init_filters_sys_path_from_leaked_pythonpath subprocess assertion excluded;
  • Ruff check, Ruff format, pre-commit, and git diff --check passed.

Regression coverage pins the exact 1,606/9,974 report, explicit and legacy threshold boundaries, zero-live-row ghost state, repair output, vector-fence behavior, and warning deduplication.

@igorls igorls added bug Something isn't working storage needs-rebase PR has merge conflicts with develop and needs rebase labels Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working needs-rebase PR has merge conflicts with develop and needs rebase storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants