Conversation
…2002) _get_client() detects a peer writer's inode/mtime change and rebuilds the client via ChromaBackend.make_client(), but chromadb caches its System (and the live in-memory HNSW segment) keyed by path. The rebuilt client is handed back the same stale segment, which on its next _persist() overwrites the on-disk index, destroying records other writers had already indexed. Observed in a live multi-writer palace: the persisted index count went backwards (4 to 3). Call the existing _force_chroma_cache_reset() on the staleness path, before make_client(), so chromadb rebuilds the segment from the on-disk state. The call is guarded by the existing inode_changed/mtime_changed check, so it has no effect on first-open. Adds test_get_client_resets_chroma_system_cache_on_reconnect, which asserts the reset runs before make_client on an mtime reconnect (fails without the fix). Refs #1963.
…n inode/mtime reopen _client() reconstructs PersistentClient on an inode/mtime change but did not drop chromadb's process-global SharedSystemClient cache first, so the rebuilt client reused the stale path-keyed System (and its in-memory HNSW segment) and could persist an outdated index over on-disk changes -- the same class as #2002, reached via _client() instead of _get_client. Add SharedSystemClient.clear_system_cache() to the external-change branch of _client(), mirroring mcp_server._force_chroma_cache_reset (#2026) and repair._close_chroma_handles. Backend-level regression test asserts the reset fires on the change reopen, strictly before the reconstruct, and not on first open (chroma-core/chroma#2536, #5843). Fixes #2028.
Two monkeypatch.setattr calls were wrapped across lines that fit within the line length; ruff format --check flagged them. Formatter-only, no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gg6g5efZ1rNbBTHqGz2Tjw
The #2002 staleness reconnect makes _get_client() call _force_chroma_cache_reset(), which clears chromadb's path-keyed SharedSystemClient cache. Two TestDeleteBySource tests grabbed a closets collection handle *before* calling tool_delete_by_source and then asserted on it afterwards, by which point the reset had dropped the Rust binding underneath the handle (AttributeError: 'RustBindingsAPI' object has no attribute 'bindings'). Re-acquire the closets collection after the tool call in both tests. Production callers already re-acquire fresh handles per call, so this is a test-lifetime issue, not a regression in the fix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gg6g5efZ1rNbBTHqGz2Tjw
#21 (CRITICAL, data-loss): multi-batch re-mine had no completion marker. A mid-file crash after batch 1 committed but before a later batch left permanently silent partial data -- the surviving drawers shared the file's unchanged on-disk mtime, so file_already_mined() treated the file as fully mined forever. Every chunk now carries chunk_total, and file_already_mined() verifies a matching-mtime group's drawer count reaches chunk_total before reporting True. Drawers with no chunk_total (legacy rows, single-shot add_drawer()) are trusted as before. #22 (HIGH, correctness/TOCTOU): source_mtime was captured via a fresh os.path.getmtime() well after content was read, chunked, and room-detected. A file appended to in that window got its new drawers stamped with an mtime that already matched the (now newer) on-disk state, so the appended tail was silently, permanently skipped on every future mine. _read_text_no_follow now returns (content, mtime) from the same fstat() that validates the file; process_file threads that single value through instead of re-stating. #23 (HIGH, silent-failure): a failed stale-drawer purge was swallowed to a debug log and mining proceeded anyway, silently producing duplicate or orphaned drawers. A purge failure now aborts this file's mine attempt (old drawers' stored mtime is untouched, so the next mine still sees a mismatch and retries) and prints a visible warning, matching every other degraded path in this module. #24 (LOW, data-loss): the old-drawer delete ran unconditionally, but the closet purge+rebuild only ran when drawers_added > 0 -- a file whose chunks all landed below min_chunk_size after boundary-splitting lost its drawers but kept stale closets pointing at now-deleted IDs. purge_file_closets now runs whenever the delete-and-rebuild cycle does, regardless of the new chunk count; only the rebuild itself stays conditional. 169 tests pass across test_miner.py/test_convo_miner*.py/test_palace.py/ test_hallways.py/test_format_miner.py/test_miner_fts5_validation.py, no regressions. Full suite: 1 unrelated pre-existing flake in test_mcp_server.py (module-global peer-writer-lock state leaking across test files in full-suite ordering -- passes standalone and as a full file; the diff here never touches mcp_server.py).
Rewrite the 3.7.0 section as short, scannable bullets like 3.6.0 — bold lead, one or two sentences per item, thematically grouped fixes — instead of multi-paragraph issue writeups.
Windows CI intermittently returns zero hybrid hits right after a fast seed write (same class as "Nothing found on disk" on tiny collections). Close the palace client after seeding so the next open re-reads flushed segments, retry search once if empty, and assert non-empty results with a clear message instead of IndexError.
…ty-results fix(tests): harden hybrid search against empty Windows Chroma reads
`os.walk` and `glob` list a FIFO, a socket and a device node as ordinary filenames, and MemPalace decides what to read from the suffix. Opening a FIFO for reading parks in the kernel until a writer appears, so a named pipe called `notes.md` in a mined directory wedged `mempalace mine` forever — no output, no error, no progress. `mine --mode convos`, `sweep`, `init`, `compress` and `split` blocked the same way. Two shapes are at fault. Four helpers already refused non-regular files with `fstat` + `S_ISREG`, but the check sat *after* a blocking `os.open`, so it could never run. Adding `O_NONBLOCK` to those opens makes the existing type check reachable: the open returns immediately and the file mode decides, with no errno guesswork. A FIFO that does have a live writer is refused just the same. Linux open(2) states the flag has no effect on regular files; the one exception is a write lease, where a non-blocking open fails EAGAIN instead of waiting out lease-break-time. Leases are granted on regular files only, so that branch re-checks the type and then opens without the flag rather than silently dropping a file that used to be mined. The rest guard with `exists()`, which is true for a pipe, and then open anyway. Those become type checks: a discovery walk drops non-regular entries before any reader sees them, and a fixed-name read decides with `is_file()` instead. `scan_project` and `scan_convos` already stat every candidate for the size limit, so the type check costs no extra syscall. Where the gate replaced an `open` that sat inside a `try`, it goes in the same `try`: `is_file()` raises `PermissionError` on a directory without `x`, which that handler already absorbed. O_NONBLOCK: miner._read_text_no_follow, convo_miner._is_regular_source_file, normalize._read_transcript_file, repair._open_regular_file_no_follow. Type gate: miner.scan_project, miner.load_config, convo_miner.scan_convos, sweeper.parse_claude_jsonl, sweeper.sweep_directory, entity_detector.detect_entities, cli._gather_origin_samples, cli._ensure_mempalace_files_gitignored, cli.cmd_compress, cli.cmd_init, project_scanner._collect_manifest_names, project_scanner._parse_gradle_root_project_name, room_detector_local.detect_rooms_local, llm_refine.collect_corpus_text, split_mega_files.main, hook_shell.count_human_messages.
docs(changelog): lead 3.7.0 with agent logstream + tighten notes
Stack the post-3.7.0 hang and silent-skip fixes for a fast patch release: - Keep #2223 (non-regular file hang) and #2088 (chunk_total / same-fstat mtime / purge abort / closet purge) as the base. - On multi-batch upsert failure, delete partial drawers and closets for that source before re-raising so the next mine retries (#2122, #2151). - Install SIGTERM/SIGHUP handlers in mcp_server.main so atexit can release the palace writer lease (#2205). - Adapt non-regular-file tests to the (content, mtime) read return type.
) Port project-miner re-mine honesty to conversation ingest so an interrupted transcript mine cannot permanently skip missing exchanges: - stamp chunk_total on every convo drawer in the pass - delete partial drawers for the source/extract_mode on upsert failure - teach prefetch_mined_set the same completeness rule as file_already_mined
…name In-place rebuild tests archive the palace directory after _seed_palace. backend.close() alone left chromadb's path-keyed System holding files open on Windows (WinError 5), so rebuild_from_sqlite aborted before the mocked upsert path and test_rebuild_from_sqlite_raises_on_upsert_failure never raised RebuildPartialError. Clear the shared cache and GC after close.
fix: 3.7.1 critical patch — hang, re-mine honesty, stale Chroma cache
Bump package, plugins, lock, OpenClaw skill, and README badge to 3.7.1. Fold Unreleased integrity notes into the 3.7.1 changelog: FIFO ingest hang, project and convo re-mine completeness, chromadb System-cache rewind, and SIGTERM/SIGHUP lease release.
chore(release): 3.7.1
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 10eeb45001
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # rebuilt client persists an outdated index over the on-disk | ||
| # change. Gated on genuine external change (not first open) so | ||
| # cold opens never pay the global-evict cost. | ||
| _clear_chroma_system_cache() |
There was a problem hiding this comment.
After any normal write changes chroma.sqlite3's mtime, the next _client() call enters this branch, clears Chroma's global system registry, and overwrites self._clients[palace_path] without closing the previously cached client. Because the old client still references its live System, the subsequent PersistentClient creates a second SQLite/HNSW system for the same palace; retained ChromaCollection handles can then keep using the first one, causing leaked locks and concurrent stale-index writers. Close and evict the cached client before clearing/reopening, as _force_chroma_cache_reset() already does.
AGENTS.md reference: AGENTS.md:L23-L23
Useful? React with 👍 / 👎.
| # the next mine. The source lock prevents this cleanup from | ||
| # deleting another miner's work for the same file. (#2122) | ||
| try: | ||
| collection.delete(where={"source_file": source_file}) |
There was a problem hiding this comment.
When a later batch upsert fails, this deletion removes every successfully committed batch after the previous complete drawer set was already purged, leaving the source with no searchable content until a future mine succeeds. The newly added chunk_total completeness check already ensures that surviving partial batches are retried rather than permanently skipped, so deleting them only turns a partial ingest failure into total temporary data loss and violates the append-safe ingest invariant.
AGENTS.md reference: AGENTS.md:L23-L23
Useful? React with 👍 / 👎.
…3.7.1 chore: merge main into develop after 3.7.0 promotion
milla-jovovich
approved these changes
Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promote
develop→mainfor the 3.7.1 releaseReleases publish from
main(see docs/RELEASING.md). This brings the 3.7.1 patch train ontomain.Version bump: #2238. Integrity stack: #2228.
Version
All release/version surfaces are 3.7.1.
Changes since v3.7.0
Bug fixes
mine,sweep,init,compress, andsplitused to block forever on a FIFO/socket that looked like a source file (suffix-based). Opens now useO_NONBLOCKso the regular-file check can run; discovery walks skip non-regular entries. (mempalace mine hangs forever on a named pipe in the mined directory (sweep, init, compress and split too) #2221)chunk_total;source_mtimecomes from the samefstatas the read; a failed stale-drawer purge aborts; closets are purged even when zero drawers remain; a mid-file upsert failure cleans partials so the next mine retries. (fix(miner): close four re-mine safety gaps in process_file #2088, A failed upsert sub-batch marks the file fully mined, so the retry never happens #2122, fix(miner): retry after partial batch upsert failure #2151)chunk_total; mid-batch failure deletes that source’s partials;prefetch_mined_setomits incomplete groups so “already filed” cannot strand missing exchanges. (mine --mode convos: file-level already-filed tracking can silently skip individual exchanges from an already-mined transcript #2183)chroma.sqlite3, MCP_get_clientandChromaBackend._clientdrop chromadb’s path-keyedSharedSystemClientcache before reopening. (_get_client() reconnect leaves the stale Chroma segment live: a peer writer's persist destroys already-indexed records (index count goes backwards) #2002, ChromaBackend._client() inode/mtime reopen rebuilds without a chromadb system-cache reset (parallel to #2002) #2028, fix(mcp_server): reset chromadb System cache on staleness reconnect (#2002) #2026, fix(chroma): reset chromadb System cache in ChromaBackend._client() on inode/mtime reopen #2032)sys.exitso the existingatexitrelease path runs (SSH disconnect, container/systemd stop). (Palace write lock not released on SIGTERM/SIGHUP (only registered via atexit) #2205)Also includes post-3.7.0 polish already on develop: #2219 changelog wording, #2220 Windows hybrid-search test harden.
After merge
main, tagv3.7.1, notes fromCHANGELOG.md[3.7.1].pypienvironment whenpublish.ymlwaits.