Changelog

All notable changes to MemPalace are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Bug Fixes

A palace with no database is no longer reported as one that passed its integrity check. sqlite_integrity_errors answers [] when chroma.sqlite3 is absent, and the MCP gate published that as checked: true, ok: true. Absence is now decided by ENOENT alone, which proves that nothing resolves under the path, and reported as the not-applicable shape #1931 introduced, checked: false/ok: null plus a reason. Every state that is not proven absent reaches the probe, and a probe that cannot open the file reports PRAGMA quick_check failed, which trips the existing -32002 refusal: a dangling symlink, a database under an unreadable directory, a symlink loop, a name the filesystem rejects, an embedded NUL in the path, and, on POSIX, a palace path whose parent is a file. A palace directory named with a byte that is not valid UTF-8 reached the probe and, up to Python 3.12, raised out of it, which mempalace mine and mempalace repair never guarded against; it is now reported like every other unreadable path. /statusz reads an absent verdict as healthy, so the new ok: null does not turn a fresh install red, and non-chroma backends stop reporting themselves unhealthy, which they had done since the #1931 fix. The size-limited startup skip still publishes a clean verdict; the only change there is that it no longer inherits the previous probe's absence reason. (#2290)

3.8.0 — 2026-08-20

Large palaces get fast and stay small: both storage backends lost their palace-wide read paths, a proxied agent session no longer loads a storage stack it never uses, and agents gained a background watcher so coordination stops stalling on nobody listening.

Features

mempalace logstream watch — a background watcher an agent can be woken by. logstream wait is a primitive, not a watcher: it caps at five minutes and reports a timeout, so every caller ended up writing the same re-arm loop and each one had to remember to carry its cursor forward. Most did not, and coordinated tasks stalled on nobody listening rather than on the work. watch owns the loop and the cursor, and exits on a match so any harness that can background a process and react to its exit gets woken — 0 when it printed a match, 2 on --idle-exit-ms, 130 when interrupted; only 0 means "you have mail". Two filters a watcher needs are things list_events cannot express, because its SQL is single-valued and positive-only: repeating a flag means "or" (--type task.request --type task.reply --type patch.ready wakes for any of them and stays quiet for everything else), and --agent <id> expands to --to-agent <id> --exclude-from-agent <id>. That exclusion is not cosmetic — to_agent=<you> deliberately also matches * broadcasts, and your own broadcasts are broadcasts, so a watcher without it wakes itself every time it posts a status. --state-file persists the cursor so a restart resumes exactly, advancing past events that were examined and rejected rather than only matches; a cursorless first run starts at the tip like the SSE live-tail rather than replaying weeks of fleet history it cannot tell is stale, and says so on stderr. --follow keeps the process alive past the first match and emits NDJSON, since repeated indented documents on one stream are not parseable JSON. The invariant throughout is that a restart may cost a duplicate, never a missed delegation. (#2315)
The monitoring protocol is documented, including the cursor rule that costs real events. Events are ordered by append order (ORDER BY rowid), not wall clock, so a peer's event is appended whenever it syncs and can already be older than a timestamp high-water mark — resuming with since_created_at therefore drops late-arriving cross-replica events permanently. Measured on a live fleet: a windows-origin event created 09:10:48Z was ingested after a mac-origin event created 09:13:21Z, in a single fifty-event window. list_events' docstring already said since_event_id was "the precise cursor … regardless of timestamp ties"; it just never reached an agent. That rule now appears in the mempalace_event_list and mempalace_event_wait tool descriptions at the point of use, both since_created_at parameters are marked "NOT a resume cursor", and coordination-protocol.md gained a monitoring section covering the modes, the announce-your-watch convention, and declaring when you are not watching — a false watcher is worse than a declared-absent one, because the requester stops looking for a human to nudge. The system-prompt snippet every agent copies was updated with it. (#2315)

Bug Fixes

sync --apply no longer deletes drawers whose source file it could not reach. _classify_drawer separated keep from remove with one Path.exists(), and missing feeds removable_ids, so every state that answered no became a deletion. Of twelve source-file states driven through sync_palace, eight were deleted and four of those had really gone: two were a file on a volume that is not mounted, at an empty mount point or at one holding a committed .gitkeep, and two were a path that could not be walked at all. Three further states ended the whole run instead, dry run included, and two of those moved with the interpreter as pathlib stopped raising. End to end, a project mined from a mounted volume lost every drawer to one sync --apply while the volume was away, and the search came back empty once it returned. Removal now asks for corroboration rather than a probe: a file not at its path is removable only while the palace still sees a source of its own, a regular file, in that same directory. A deletion leaves the file's neighbours where they were; an unmounted volume takes all of them at once, and no errno separates the two. Both halves of the verdict are read again at the moment it is formed, because a volume can leave inside one pass and come back inside one. Everything uncorroborated lands in a new bucket, unresolved, never added to removable_ids and reported the way removals are: counted beside the other buckets, and its source files named in unresolved_by_source and printed, with the remainder stated when that list is cut at five. The price: a directory with no surviving known source corroborates nothing, so a file deleted alone from a one-file directory, or a directory's files deleted together, is kept and reported rather than pruned. (#2320)
list_drawers and the tunnel tools stop cold-loading the vector index on a chroma palace. Both paged col.get(), which opens the collection and loads HNSW before answering a question that is pure metadata: on a 1.7 GB / 165k-drawer palace list_drawers with no filter and limit=20 took 36.6 s, and find_tunnels died at 29.7 s with an internal tool error. chroma.sqlite3 already holds everything those calls need, so they now read it directly and never open the collection. Two traps had to be avoided to make that a win rather than a trade. chroma:document lives in embedding_metadata alongside the loci, so an unqualified join drags the palace's entire verbatim text — 162 MB across 331k rows here — into memory to render one page; documents are excluded from the scan and the displayed page is hydrated separately. And embedding_id is only indexed as the second column of UNIQUE (segment_id, embedding_id), so a join filtering on it alone degenerates into a full scan of embedding_metadata: 5.7 s for a single twenty-row page, against 0.003 s once the segment is sought first and embedding_metadata's (id, key) primary key second. The wing/room filter is pushed into SQL rather than applied afterwards. Measured on the same palace, list_drawers(wing=…, limit=20): 2.31 s and 1148 MB peak RSS → 0.01 s and 86 MB. find_tunnels' recent field survives the move via a MAX(date) on the grouped read, and a missing or unreadable palace still reports a diagnostic rather than looking like a palace with no tunnels. (#2314)
sqlite_exact search and status stop scanning the whole palace. query() was exact cosine over every row — selecting id, document, metadata_json and embedding, JSON-parsing metadata, then dotting in a Python loop — which mempalace_search does twice, once for drawers and once for closets. status paged every metadata row because the backend had no facet_counts and the taxonomy read was chroma-only. Ranking now runs vectorized over the embedding column with only the top-k documents hydrated, the embedding matrix and the wing / room / source_file columns are cached on the long-lived backend handle and invalidated on write, and facet_counts plus a grouped json_extract read serve status, list_wings and list_rooms. Exact cosine ranking is unchanged. Measured on a live 167k-drawer / 166k-closet palace: mempalace_status 6997 ms → 1045 ms, warm mempalace_search 6364 ms → 210–1600 ms, wing-scoped search 3910 ms → 1453 ms. (#2308)
The remaining palace-wide reads on sqlite_exact are gone. get(ids=…) scanned every row and then did a dict lookup, include=["metadatas"] still selected documents and embeddings, and equality filters with LIMIT/OFFSET were applied in Python — so list_drawers at limit=20 took 2752 ms because it loaded every document in the palace to show twenty previews. Those now go to SQL, pagination walks metadata only to collapse logical drawers and compute the total, and the page's previews are hydrated by id. status, list_wings, list_rooms and get_taxonomy share one grouped result behind a 5 s cache dropped on writes, and graph_stats gained the same sqlite reconstruction the chroma path uses. (#2311)
wing, room and hall are indexed places rather than JSON scanned palace-wide. sqlite_exact stored them only inside metadata_json, so every structural question — taxonomy, graph_stats, list_drawers where=wing, facet counts — meant json_extract across the corpus. They are now VIRTUAL generated columns over the same JSON with a composite (collection_id, wing, room, hall) index, which existing palaces pick up on their next writable open. VIRTUAL is the point: the 1.6 GB embedding blob table is not rewritten, so the migration walks metadata once rather than copying a table. Read-only opens against a pre-migration palace fall back to json_extract unchanged. (#2313)
embeddinggemma no longer produces silently unusable vectors on Apple Silicon. With the default embedding_device=auto, _AUTO_ORDER puts CoreML ahead of CPU, and CoreML returns NaN or all-zero vectors for this model without raising — ONNX Runtime only warns on stderr about how few nodes it could take. Same model and input: CoreML gave 9216 NaN in the hidden state and a 768-zero pooled vector with norm 0.0, against a healthy norm on CPU, reproduced under two onnxruntime versions and two graph partitionings. Nobody opted into this, and a repair rebuild-index under those conditions rewrites the whole palace with vectors that are indistinguishable from healthy ones once stored. Two layers now: a per-model provider denylist keeps auto from selecting CoreML for embeddinggemma at all, and the vectors are checked for a finite and non-zero norm before use — an all-zero vector passes any isnan() test and is exactly as unusable, so a NaN-only check would have looked like a fix while missing the observed case. Anyone auditing an existing palace should look for zero-norm vectors, not NaN. CUDA and DirectML are untouched, and other models were never affected. (#2283)
A proxied MCP session no longer loads the storage stack it never uses. mempalace-mcp is spawned once per agent session, and whenever a hub is running every one of those processes is a pure proxy — _dispatch_stdio_request forwards each JSON-RPC request over HTTP and local storage is never touched. Importing mempalace.mcp_server to do that cost ~77 MB regardless, because chromadb (~61 MB by itself), numpy, pydantic, grpc and opentelemetry are all imported at module scope: a fleet of 50 agents spent ~3.9 GB holding proxies that did no work. The mempalace-mcp entry point is now mempalace.mcp_proxy, which imports only the standard library plus config and server_registry; a proxied session runs at ~17 MB measured end to end (~24 MB import weight against ~80 MB), and the full server is imported lazily the first time this process must answer a request itself. Anything that is not a plain stdio session — another transport, a server-side flag, an unrecognised argument — goes straight to the full server unchanged. The local fallback is preserved exactly, including the rule that a mutating call which failed mid-flight is never replayed locally, and it now says so: the first tools/call served locally carries a notice in result.content telling the driving agent its memory backend lost the hub and this process is holding the index. Restoring stdout on that lazy import is required, not cosmetic — importing the server performs os.dup2(2, 1), so responses would otherwise be written to stderr and the session would hang. (#2312)
A long-running MCP server no longer grows by ~440 MB per collection open. ChromaBackend._client keys its client cache on chroma.sqlite3's inode and mtime to detect writes by another process, but constructing a chromadb.PersistentClient writes to that file itself, and so do the collection opens that follow — so the stamp taken at construction time was already stale by the time the next open compared against it. Every search opens mempalace_drawers and then mempalace_closets, and the first open moved the mtime the second one checked, so the cache missed on essentially every request: each search rebuilt the client and reloaded both HNSW segments, and the displaced client was dropped from the dict without being closed, so its native index memory was never returned. On a palace of ~165k vectors that was ~650 MB of resident growth per search; a server left running for an afternoon reached 3.9 GB. The freshness stat is now re-baselined once the backend's own operation finishes — including writes through ChromaCollection, so the file-a-drawer-then-search cycle stops reloading the index too — which makes the recorded value mean "chroma.sqlite3 as this backend last left it", and a genuine external change still rebuilds. The rebuild path also closes the client it replaces. Measured over eight searches on the same palace: 951 MB → 3522 MB before, 940 MB → 965 MB after. An external write landing while one of our own operations is in flight is absorbed into the new stamp and picked up on the next change; mtime cannot distinguish writers, and PRAGMA data_version does not help because chromadb's own connection reads as foreign to a probe connection. (#2307)
mempalace_mesh_peers answers with the real estate from every transport, not just the hub. The peer sync loop is started only by _serve_http, and the estate it builds (_PEER_SYNC_STATE, _KNOWN_PROFILES) lives in process memory — but the tool that reports it ships in every transport, including the stdio servers agents actually connect through. Those processes never run a sync round, so they answered from a permanently empty estate: each peer reduced to a bare name and url with no reachable, last_success_at, remote_version_vector or profile, origin_profiles holding only this node, and configured peers misreported as unnamed_origins because their replica_id was never learned — all while the hub next door had the complete picture. The sync loop now publishes the estate to mesh_state.json in the per-palace server state directory (0600, write-and-rename) after every round, and _mesh_peers_payload merges it underneath any in-process state, so the syncing process still trusts its own fresher reading. A new estate_source field says whether the status was observed in this process or published by the hub, when it was published, and whether that hub is still alive — a crashed hub leaves a last-known-good estate that is shown but never read as live. peers.json tokens never reach the file. (#2309)
sweep books a failed stat as a failure again. The non-regular-file gate added in 3.7.1 reads stat.S_ISREG(f.stat().st_mode) inside a try, and its except OSError printed SKIP and moved on. A dangling symlink, a symlink loop and a file unlinked between rglob and the gate all raise there, and before the gate existed every one of them reached sweep() and was booked in failures — so sweep went from reporting a transcript it could not read to reporting success. A failed probe is now an error, not a benign file type: it is logged, printed as WARNING, and appended to failures, while a probe that succeeds and says "not regular" still skips silently. (#2221)
mempalace init no longer tracebacks on a directory it cannot enter. _parse_gradle's is_file() gate sat in front of the try that the parser's own except OSError provides, so a manifest under a directory with r but no x raised PermissionError out of a call that used to answer "no manifest name". The gate moved inside that try, and _collect_manifest_names stats through os.path.isfile, which reports rather than raises. (#2221)
split no longer blocks on a FIFO at its own output name, nor writes through a broken link. The type gate in main covers the files the glob listed; split_file builds its output names itself, so a pre-existing named pipe at one of them wedged write_text in the kernel waiting for a reader. The check asks about the link itself rather than its target, because a dangling symlink at an output name reads as "nothing there" and write_text would create the target — landing a chunk outside the output directory. Output names that are anything but a regular file are now skipped with a SKIP line. (#2221)
The FTS5 auto-heal checks the content table before it rebuilds from it. PRAGMA quick_check's isolated malformed inverted index for FTS5 table says the inverted index and embedding_fulltext_search_content disagree, not which of them is wrong, and damaging the content table produces that same wording on SQLite 3.45.1, 3.47.1 and 3.51.2 alike. The heal rebuilt from that table regardless and reported "rebuilt from intact content", so a damaged content table cost the palace its lexical reach — 12 of 30 drawers stopped answering lexical_search for a word embedding_metadata still held — permanently on the mine path, where nothing re-files afterwards. Chroma writes every document twice, into embedding_metadata under chroma:document and into the FTS5 table at rowid = embeddings.id, so the shadow copy has an authority: the heal now checks it against that table, restores the rows that disagree and rebuilds, all in one transaction under the mine lock. Rows the authority cannot speak for keep their content and are named in the output, and a check that cannot conclude declines the rebuild instead of guessing. (#2278)

3.7.1 — 2026-08-12

Post-3.7.0 integrity patch: ingest no longer hangs on non-regular files, incomplete mines can be retried instead of permanently skipped, chromadb reconnect no longer rewinds the HNSW index, and MCP releases the writer lease on SIGTERM/SIGHUP.

Bug Fixes

Ingest commands no longer hang on a named pipe. os.walk and glob list a FIFO as an ordinary filename and MemPalace decides what to read from the suffix, so a pipe called notes.md in a mined directory wedged mine in the kernel forever: opening a FIFO for reading waits for a writer that never arrives, and the S_ISREG refusal written on the next line could never run. mine --mode convos, sweep, init, compress and split blocked the same way through their own readers. The four affected opens now pass O_NONBLOCK, which makes the existing type check reachable — a pipe is refused on its mode, with or without a live writer — and the discovery walks drop non-regular entries with a SKIP: <name> (not a regular file) line, so the readers that use a plain open() never see one. Regular files read back byte-identical; the one case where the flag is not inert, a reader breaking a write lease, re-checks the file type and retries without it rather than dropping the file. mine --mode extract was already immune through its zero-size gate. (#2221)
Project re-mine no longer silently skips a partial or interrupted file. Four related gaps in process_file: (1) multi-batch upserts now stamp every drawer with chunk_total so file_already_mined can tell "N of N committed" from "crashed after batch 1"; (2) source_mtime comes from the same fstat as the content read, so an append between read and a later re-stat cannot permanently hide the new tail; (3) a failed stale-drawer purge aborts the file instead of half-overwriting; (4) closets are purged even when the re-mine ends with zero drawers. A mid-file upsert failure also deletes the partial drawers and closets for that source before re-raising, so the next mine retries instead of treating the incomplete set as complete. (#2088, #2122, #2151)
Conversation mine completeness matches the project path. Convo drawers now stamp chunk_total; a mid-batch upsert failure deletes that source's partial drawers before re-raising; prefetch_mined_set omits incomplete groups so the bulk "already filed" skip cannot permanently strand missing exchanges from an interrupted transcript mine. (#2183)
Stale chromadb System cache is cleared on palace reconnect. After a peer or rebuild changes chroma.sqlite3 on disk, both mcp_server._get_client and ChromaBackend._client drop chromadb's path-keyed SharedSystemClient cache before reopening — otherwise the stale in-memory HNSW segment is reused and can persist an outdated index over the peer's writes (index count going backwards). (#2002, #2028, #2026, #2032)
MCP releases the palace writer lease on SIGTERM/SIGHUP. The lease was only released via atexit, which CPython skips on those signals' default disposition. SSH disconnect (SIGHUP) and container/systemd stop (SIGTERM) therefore left mine_palace_*.lock naming a dead PID until a contender's liveness check reclaimed it. main() now installs handlers that exit through sys.exit, so the existing atexit release path runs. (#2205)

3.7.0 — 2026-08-11

Features

Agent logstream coordination (RFC 003). Append-only event layer for multi-agent work: durable task packets, wait/ack handoffs, patch and file artifacts, and live tailing over the MCP HTTP hub (GET /logstream/stream SSE). MCP tools include mempalace_event_append / list / wait / ack, artifact put/get, and patch submit; CLI mempalace logstream mirrors the core verbs. Events and artifacts stay local, verbatim, and separate from the drawer palace (logstream.sqlite3). Shared-brain / multi-machine agent fleets no longer need a human to relay status between hosts. (#2162, phases 1–5)
Logstream multi-master sync foundation (RFC 004 step 0). Estate-level logstream replication hooks so coordinated agents can share the coordination layer across palace replicas — the storage step for a replicated shared brain. (#2162, logsync)
OpenAI-compatible embeddings. Opt-in embedding_model: "openai-compat" talks to any /v1/embeddings endpoint (LM Studio, llama.cpp, vLLM, Ollama's OpenAI shim, or a self-hosted server) over stdlib urllib — no new dependency. Config and env vars set URL, model, and key; the model id is part of the embedder name so a model change forces a reindex. Default MiniLM/ONNX path is unchanged. (#1671, #1559)
Search date window. mempalace_search and mempalace search accept since / before ([since, before) on filed_at), shared with list_drawers via mempalace.date_window. Undated drawers are excluded while a bound is active; the vector candidate pool widens under a filter and reports truncation when the pool is full. (#2000, #463)
Hermes memory provider (core). In-package Hermes MemoryProvider files live turns through file_conversation_exchange() so metadata matches convo mining, with a background worker so the agent loop never blocks. Install/backfill/docs remain a stacked follow-up. (#1915, #2215)
RFC 002 source adapters on mine. mempalace mine <source> --source <adapter> resolves registered adapters, holds the palace writer lease for the full ingest, and keeps dry-runs inert. Legacy --mode paths are unchanged. (#2068, #2062)
Hook write-routing through the daemon. Background hook saves and mines can honor the shared write-routing policy so multi-session setups serialize mutations through one local owner. (#2030, #1963)

Performance

EmbeddingGemma groups documents by size before sub-batching, cutting padded-token work on long sweeps without changing vector meaning. Applies only to embedding_model: embeddinggemma. (#2104)
HNSW capacity probes are cached and invalidated by palace file signature, so repeated status/taxonomy paths no longer re-scan native segments every call. (#2051, #1471)
chunk_text line numbering is O(N), fixing multi-second hangs on large sources. (#2054, #2055)

Bug Fixes

Mining works again on the default Chroma backend. Explicit-embedding writes no longer hand Chroma np.float32 scalars that normalize_embeddings rejects. (#2187)
ChatGPT data exports are parsed as conversations, not stored as raw JSON arrays. (#2160)
Local backends enforce process-lifetime single-writer ownership. File-backed palaces require one writer owner for the process lifetime; read-only MCP may coexist; remote backends remain multi-process. (#2079, #2045)
MCP refuses writes when the served library drifts (mempalace, and chromadb when that backend is active) after an upgrade without restart. Opt out with MEMPALACE_MCP_ALLOW_STALE_LIBRARY. (#2081, #899)
Chroma HNSW write defaults match chromadb (batch_size=100 / sync_threshold=1000), retiring the old 2/2 bloat guard. (#2107, #2106)
Repair and recovery are safer under contention. Mine-lock before archive, preserve temp collections on failed swap, fail loud on truncated pagination; dry-run is a true preview in both legacy and from-sqlite modes; backups skip sockets/pipes/device nodes so a live palace socket no longer aborts repair/migrate. (#2109, #2086, #2087, #2133, #2144, #2207, #2212)
rebuild_index holds the palace writer lease for the full snapshot→rebuild/swap cycle. (#2195)
Orphaned per-source mine locks are reaped (age + nonblocking flock), throttled to once per 15 minutes; palace-level locks are untouched. (#2200)
HNSW divergence is preflighted before remaining count() crash sites across mine, dedup, migrate, repair, and palace helpers. (#2093)
Re-mine and conversation ingest no longer lose or duplicate drawers. Content-hash dedup, sweeper purge scope, round-trippable drawer_id on search; Claude Code subagents/ skipped by default (--include-subagents to opt in). (#2050, #2125, #2090, #1330, #1217)
MCP and daemon lifecycle harden multi-agent use. Read-only refuses config/checkpoint-ack host mutations; stdio exits on EOF; daemon defers lock-refused jobs; Windows stdout capture falls back or fails closed if the protocol stream cannot be restored. (#2126, #2072, #2029, #2211, #2210)
Encoding and Windows locale hardening. Pin UTF-8 on config/dialect opens; mojibake repair no longer destroys clean Portuguese/Vietnamese/Turkish prose; repair-encoding CLI reconfigures stdio; non-ASCII CLI symbols replaced for GBK consoles. (#2098, #2208, #2194, #1104, #1034, #2193)
Small correctness fixes. Entity-candidate ReDoS guard; reject pathological chunk_overlap; worktree cwd maps to the project wing; markdown emphasis is not emotion; service entrypoints restore MEMPALACE_PALACE_PATH; systemd Restart=always with idle watchdog docs; valid docker-compose.yml environment key. (#2127, #2056, #2206, #2199, #2192, #2203, #2204, #2188)

Documentation

Agent logstream concept page, coordination protocol, shared-brain fleet guide, and RFC 003/004. (#2162)
Operator write-routing / single-writer recovery notes. (#2079)
Remote-server idle watchdog and read-only semantics. (#2126, #2204)
README Docker section leads with the published image and real mount/permission pitfalls. (#2196)
MX3 public-shim example and CONTRIBUTING Discussions cleanup. (#597, #555)

Internal

Docker publish is gated on a real smoke script (Compose parse, mine, MCP handshake). (#2189)
Embedding empty-batch / plain-sequence regression tests; HNSW defaults assertions. (#2191, #2159)

3.6.0 — 2026-07-14

Features

Turnkey secure remote / team server. mempalace serve exposes the full MCP surface over HTTP with secure defaults: non-loopback binds require a bearer token, generated tokens are stored with restrictive permissions and passed through the child environment rather than argv, native TLS 1.2+ is supported, and --read-only both hides and refuses mutating tools. Docker Compose, systemd, and environment templates are included for deployment. (#1877, #1897, #1900)
Milvus storage backend. The new opt-in milvus backend supports embedded Milvus Lite, self-hosted Milvus, and Zilliz Cloud through pymilvus, with COSINE vector search, native BM25 lexical search for new collections, namespace isolation, target-mismatch protection, and an optional milvus dependency extra. (#1899)
Atomic knowledge-graph fact replacement. supersede() / mempalace_kg_supersede closes an open fact and opens its successor at one shared instant, so model, employer, address, and other single-valued facts can change without a hand-written invalidate/add race. Temporal queries now use half-open intervals at timestamp precision, returning only the successor at the transition boundary while preserving whole-day semantics for date-only facts. (#1913)
Conversation chronology is preserved. Conversation drawers now retain transcript time as authored_at alongside ingest time, search results expose it, and exact hybrid-score ties prefer the more recently authored drawer. An idempotent, dry-run-first backfill script adds the metadata to existing palaces without re-embedding. mempalace_list_drawers also accepts inclusive since and exclusive before filing-time bounds. (#1890, #1891)
Mined sessions populate the associative graph. A precision-biased, no-LLM structural extractor records code symbols, URLs, paths, and qualified identifiers as drawer entities; conversation mining then derives hallways from them. The new mempalace hallways CLI command exposes the graph. (#1894, #1895)
Per-project mining exclusions. exclude_patterns in mempalace.yaml filters pre-scanned project files without changing the corrected --limit semantics. (#1213, #1953)
LaTeX project coverage. .tex and .bib files are now treated as readable prose sources. (#1901)

Performance

MCP initialization no longer waits for palace-wide integrity work. stdio answers the JSON-RPC initialize request immediately while startup preflight runs on a background thread; a tool arriving mid-probe joins the same verdict instead of launching duplicate work. The startup SQLite probe is also skipped above a configurable size limit (512 MiB by default), while repair preflights remain strict. (#1911, #1987)
Qdrant metadata counts use server-side facets. Taxonomy, overview, and status paths can count metadata remotely instead of fetching and tallying every record; the embedding wrapper now forwards the facet and bulk-metadata capabilities correctly. (#1868, #1898)
pgvector metadata-only fetches omit document payloads, reducing transfer and decoding work on enumeration paths. (#1892)

Bug Fixes

Chroma recovery distinguishes derived-index damage from data loss. Valid all-layer-0 HNSW segments with an empty link_lists.bin are no longer quarantined repeatedly; isolated FTS5 inverted-index corruption is rebuilt from intact content during repair and after mining; embedded NUL bytes are sanitized before reaching ChromaDB. (#1716, #1872, #1878, #1927, #1928)
SQLite recovery is bounded, atomic, and honest. Integrity checks wait up to 15 seconds for transient writer contention instead of reporting a healthy palace as corrupt. In-place recovery archives use atomic os.rename rather than shutil.move's destructive copy/delete fallback. repair --mode from-sqlite now rebuilds FTS5, vacuums, and requires a clean final PRAGMA quick_check; cleanup failures return non-zero with recovery details instead of printing a false success banner. (#1945, #2015, #2017)
CLI search checks HNSW divergence before opening ChromaDB. A diverged palace routes directly to the existing SQLite BM25 fallback, avoiding embedder or collection initialization against damaged native index state while leaving healthy Chroma and non-Chroma backends on their normal vector path. (#2016)
Writer leases recover instead of stranding servers read-only. A server refused while a peer owns the palace now retries on each mutating call and promotes itself after the peer exits. Status remains read-only and cannot acquire the writer lease, the in-process re-entrant holder set is released even across asynchronous interruption, and checkpoint / delete_by_source are correctly classified as mutations for both read-only serving and peer-writer protection. (#1923, #1930, #1934, #1960, #1970, #1971)
Status reports only checks that actually ran. Non-Chroma backends now mark the Chroma SQLite integrity check as not applicable rather than claiming an unperformed success. (#1931, #1946)
Conversation transcripts are treated as mutable. Appended or rewritten sessions are purged and re-filed based on modification time, preventing later turns from being silently skipped. Claude Code tool-results/ sidecars are excluded from conversation scans so raw machine dumps cannot flood the embedding space. (#1957, #2010)
Explicit palace selection now scopes derived graph state. Project, conversation, and format mining write hallways and tunnels beside the selected --palace instead of leaking them into the ambient default palace. (#2018)
Remote and local server hardening. Non-loopback HTTP now requires a token unless an explicit insecure override is supplied; optional-provider probes cannot reuse an ambient external key before the user selects that provider; hook transcript paths and file-copy/repair paths receive stricter local validation and no-follow handling. Importing the MCP server no longer clobbers the host application's root logger, HTTP writes can re-enter the process-wide palace lock safely, and routine client disconnects no longer emit server tracebacks. (#1859, #1860, #1864, #1885, #2003, #2004)
Mining and configuration correctness. Oversized files produce a visible stderr warning, ~ in configured palace paths expands consistently, wing slugs handle special characters, and Windows background daemon / synchronous hook mines use CREATE_NO_WINDOW. (#923, #1852, #1857, #1863, #1865)
mempalace init handles non-ASCII .gitignore files on Windows. The project-file ignore guard now reads and appends UTF-8 explicitly instead of relying on locale defaults such as GBK. (#1648)
L1 wake-up surfaces the latest moments. Drawers with equal importance are now ordered by filed_at recency rather than insertion order, so startup context prefers recent memories instead of the oldest ones. (#1630)
Backend detection requires a SQLite magic header before classifying a target as Chroma or sqlite_exact, preventing unrelated files from being mistaken for a palace. (#1893, #1896)

Documentation

Added a storage-backend configuration reference, a remote/team-server deployment guide, authored_at migration guidance, and refreshed the OpenClaw integration for the full 36-tool MCP surface. (#1719, #1877, #1890, #1904, #1905)

Internal

Expanded WAL crash-safety, idempotence, and redaction-path coverage (#1869); updated GitHub Actions and Ruff; repaired uv.lock drift so it again matches the Ruff 0.15.20 pin and includes Python 3.9 dependency markers introduced by the Milvus resolution.

3.5.0 — 2026-06-22

Features

Opt-in local daemon for queued writes. A new mempalace daemon queues MemPalace writes through a single local process so background mines, diary saves, and hook-driven ingests serialize against one palace handle instead of racing for it. Opt-in and local-only — nothing binds to a public interface. (#1826)
Opt-in HTTP transport for the MCP server. mempalace-mcp --transport http serves JSON-RPC at POST /mcp (with a GET /healthz liveness probe) for operators running MemPalace behind a long-lived HTTP MCP client/proxy, avoiding the long-lived-stdio framing failures of #1801. stdio remains the default and is unchanged. The transport reuses the exact stdio request dispatcher (no separate write/search path), binds 127.0.0.1 by default, and is hardened against the two ways a local HTTP server leaks to the network: it pins the Host header to loopback on a loopback bind and rejects any non-loopback browser Origin (DNS-rebinding/SSRF guard), and supports an optional bearer token via MEMPALACE_MCP_HTTP_TOKEN (required on /mcp, never on /healthz). A 16 MiB request cap and a loud warning when bound to a non-loopback host round it out. (#1801, #1806)
mempalace_checkpoint batch-save MCP tool. Collapses multiple add_drawer calls plus an optional diary entry into a single MCP round-trip for agents that want to file a whole session at once. Stores content verbatim and reuses the existing idempotent add/dedup path. (#1851)
mempalace_delete_by_source bulk-cleanup MCP tool. Exact-match, dry-run-by-default deletion of every drawer (and its matching closet/AAAK index entries) for a given source_file — the recourse for benchmark/eval files mined into the same wing as real data and drowning out search. The dry run reports the drawer and closet blast radius before anything is removed, and the commit writes a WAL audit entry. (#1722, #1729)
Optional source_file filter for mempalace_search. Scope a search to an exact stored source path. The filter is threaded through every search path (vector, BM25/SQLite fallback, lexical union, and index-mismatch fallback) so it never silently drops a matching drawer, and results now expose the full source_path as a round-trippable key. (#1815, #1817)
New transcript parsers / importers. Continue.dev session parser (#731), Gemini CLI / AI Studio JSON session import (#204), and a Pi agent JSONL session normalizer (#169).
Wider miner language coverage. C# / .NET, PHP (#1819), Swift / Kotlin (#1368), and Java project detection including rootless subprojects (#1720).
Final mine on Claude plugin SessionEnd. The Claude Code plugin now runs a closing mine when a session ends so the last exchanges are captured without waiting for the next save nudge. (#1814, #1820)

Performance

Overview/status MCP tools answer from the SQLite aggregate. Large palaces no longer time out building wing/room/status overviews — the counts come from a single SQLite aggregate instead of a client-side fetch-and-tally. (#1748, #1379)
graph_stats SQLite fast path. Knowledge-graph stats are computed in SQLite rather than walking the collection, fixing large-palace timeouts. (#1379)
Embedder caps ONNX-runtime intra-op threads so a background mine no longer pins every core. (#1068)
Backend pagination pushed into the query. sqlite_exact (#1841, #1842) and pgvector (#1830, #1840) now apply get(limit, offset) in SQL, and Qdrant fetches bulk metadata in a single scroll with a larger page size (#1796, #1832).

Bug Fixes

pgvector tolerates hostile transcript bytes. A lone Unicode surrogate (#1833) or a NUL byte (#1829) in a transcript no longer aborts the whole mine — both are sanitized before the row is written.
SQLite read-only URIs are percent-encoded so palace paths with spaces or special characters open correctly, and _sqlite_graph_stats is routed through the same sqlite_read_uri helper.
Stale ChromaDB HNSW divergence routes to the SQLite fallback instead of failing the read outright. (#1816, #1822)
Diverged-index recovery now points at repair --mode from-sqlite, not a re-mine. A failed ChromaDB HNSW compaction leaves the index out of sync while the rows stay intact in chroma.sqlite3; the old "re-mine from source" advice silently dropped MCP-added drawers and diary entries (which have no source file). Both the legacy repair/rebuild_index error messages and the repair-status recommendation, plus the recall skill docs, now guide users to rebuild from SQLite. (#1843, #1847, #1849)
The MCP server refuses a second writer for the same palace rather than letting two processes race the same HNSW handle. (#1818, #1823)
Windows hook miner spawns with CREATE_NO_WINDOW so background mines no longer flash a console window. (#1783, #1848)
fact_checker __main__ no longer emits a runpy warning under the test runner. (#1798)

Internal

Live-substrate conformance test module for pgvector (#1769); dependabot bumps for docker/login-action (3→4), docker/build-push-action (6→7), and docker/metadata-action (5→6) (#1788, #1787, #1786); ruff dev dependency bumped to 0.15.18.

3.4.1 — 2026-06-14

Features

Cursor IDE plugin (.cursor-plugin/). Drops into ~/.cursor/plugins/local/mempalace (or installs from the Cursor marketplace once published) and auto-registers the mempalace-mcp server, five slash commands (/mempalace-help, /mempalace-init, /mempalace-mine, /mempalace-search, /mempalace-status), and the model-invocable mempalace skill — no manual ~/.cursor/mcp.json edit required. The plugin manifest deliberately omits a hardcoded version field — mempalace/version.py is the single source of truth, so there is nothing to drift on the next release (a contract test enforces the field stays absent). The canonical plugin components (commands/, skills/, mcp.json) are real files at the plugin root; no symlinks are committed (committed symlinks materialise as broken text files on Windows clones with core.symlinks=false). Mirrors the surface of .claude-plugin/ and .codex-plugin/ without duplicating their hook scripts: the Cursor hook scripts under hooks/cursor/ (shipped in the same release) remain the canonical install path for stop / preCompact / sessionStart, wired separately by hooks/cursor/install.sh. Contract tests in tests/test_cursor_plugin_manifest.py cover manifest JSON validity, kebab-case naming, ..-free relative paths, on-disk path resolution, marketplace alignment, MCP config shape (mcpServers wrapper required by Cursor, unlike Claude's flat .mcp.json), the version-field-absent guard, the no-symlink guard, and every skill/command frontmatter — all pure file inspection so they run on any CI platform without Cursor itself.
Cursor IDE hook support (stop / preCompact / sessionStart). Three new bash hooks live under hooks/cursor/ and share a lib/common.sh helpers module. The save hook counts stop invocations per Cursor conversation_id and emits a followup_message every MEMPAL_SAVE_INTERVAL (default 15) so the agent files the session into MemPalace and writes a diary entry. Unlike the silent-by-default Claude Code hook, the Cursor followup fires on by default: Cursor's transcript format is undocumented and normalize.py has no Cursor parser yet, so the background mempalace mine --mode convos is best-effort only and the followup_message is the load-bearing verbatim-capture path. Users who want the Claude-style "zero tokens in the chat window" behaviour can suppress it with MEMPAL_CURSOR_SILENT=1 (or MEMPAL_VERBOSE=false); the default flips to silent once a Cursor transcript parser lands. The precompact hook synchronously mines the transcript before Cursor's compaction summarises it and drops a marker so the next stop forces a save nudge (Cursor's preCompact is observational-only — it cannot block or emit a followup_message, unlike Claude Code's PreCompact); the synchronous mine is bounded by Cursor's per-hook timeout, and because mempalace mine is incremental/append-only a killed mine resumes cleanly on the next run rather than corrupting the palace. The wake hook is Cursor-only: sessionStart returns additional_context telling the agent to recall scoped to the wing inferred from the workspace root. Honours the same MEMPALACE_HOOKS_AUTO_SAVE=false kill switch as the Claude Code hooks, plus a new MEMPAL_DISABLE_HOOK=1 alias and a MEMPAL_STATE_DIR env override. Per-conversation state files are garbage-collected by a daily-throttled, Cursor-namespaced TTL sweep (MEMPAL_STATE_TTL_DAYS, default 30) so cursor_*.count / cursor_*.pending cannot grow unbounded — shared logs and other editors' state are never touched. Includes an opt-in installer at hooks/cursor/install.sh with --scope user|project, --variant full|minimal, --dry-run, and --uninstall (idempotent, preserves unrelated hooks via python3-based JSON merge — no jq dependency). Example wirings live at examples/cursor/hooks.json and examples/cursor/hooks.minimal.json; they are intentionally not placed at the repo root because Cursor auto-loads project hooks from any trusted workspace and we do not arm hooks on contributor checkout. Per-event stdin/stdout schema documented at hooks/cursor/STDIN_SHAPE.md. Walkthrough at website/guide/cursor-hooks.md. Coverage added in tests/test_cursor_hooks_shell.py and tests/test_cursor_hooks_install.py.
First-class Antigravity IDE support. New .antigravity-plugin/ package + idempotent installer at hooks/antigravity/install.sh that registers MemPalace as a Google Antigravity plugin (MCP server, skill, two lifecycle hooks) at ~/.gemini/config/plugins/mempalace/. The Stop hook background-mines the active conversation transcript every Nth fire (default 15, configurable via MEMPAL_SAVE_INTERVAL); the PreInvocation hook injects verbatim memory on the first model call only via Antigravity's injectSteps[].ephemeralMessage output, gated by invocationNum == 1. Both hooks are bash 3.2.57 compatible (macOS default), use the same ~/.mempalace/hook_state/ directory as the Claude Code / Codex / Cursor hooks (antigravity_*-namespaced state files), and respect every existing kill switch (MEMPAL_DISABLE_HOOK, MEMPALACE_HOOKS_AUTO_SAVE, ~/.mempalace/config.json hooks.auto_save). Installer is cmp-gated (re-run produces a byte-identical install), uninstall is basename-guarded (refuses to wipe a directory whose basename isn't mempalace), and --dry-run is side-effect free. Full audit of which Antigravity surfaces we ship and which we deliberately don't is in hooks/antigravity/INVESTIGATION.md. User-facing guide: website/guide/antigravity.md. Standalone examples in examples/antigravity/.
- Zero-config interpreter resolution. mempal_resolve_python now derives the Python interpreter from the mempalace-mcp / mempalace console-script shebang on $PATH before falling back to python3. The common uv tool install mempalace / pipx install layout installs the console scripts into an isolated environment whose interpreter is not system python3, so the previous command -v python3 resolution landed on a Python that couldn't import mempalace, the -m mempalace probe failed, and mining silently never fired. Resolution is pure shebang parsing + stat (no Python subprocess at source time, preserving the hook performance budget). MEMPAL_PYTHON remains the explicit override. Documented under How the hooks find your mempalace install in the guide.

Bug Fixes

embeddinggemma no longer OOM-kills bulk re-embeds. EmbeddinggemmaONNX.__call__ ran a single session.run over its entire input, so a repair-scale batch (5000 docs) allocated attention buffers far beyond available RAM and the kernel killed the process silently: mempalace repair --yes on an embedding_model: embeddinggemma palace died right after Building temporary collection: with no traceback and no crash report (#1770). Embedding now runs in sub-batches of 32 docs (constructor-tunable batch_size), matching the internal batching of ChromaDB's bundled MiniLM embedder. Per-document vectors are unchanged: the model's pooled output is attention-masked, so sub-batch padding does not affect values. The embedder is also hardened for shared use: the process-wide EF cache and the lazy model load are thread-safe (concurrent first calls build exactly one ONNX session), and __call__ handles a bare string, None, and empty input without triggering the model download.
Backup retention to prevent unbounded disk usage. mempalace migrate (full-palace <palace>.pre-migrate.<timestamp> copies) and mempalace repair max-seq-id (chroma.sqlite3.max-seq-id-backup-<timestamp> copies) each wrote a fresh, full-size, timestamped backup every run and never deleted the old ones. On a machine that mines or repairs on a schedule, those copies could silently accumulate until they filled the disk — one palace was found with hundreds of GB of stale backups beside a few hundred MB of live data, hidden from a normal du of the home directory. A new max_backups setting (default 10, env MEMPALACE_MAX_BACKUPS, or config.json) now prunes the oldest backups after each new one is written. Set it to 0 to keep every backup. Pruning is keyed by filesystem mtime, scoped strictly to each backup's own naming pattern (live data is never touched), and best-effort so a deletion failure can never abort a migration or repair that already succeeded.

3.3.6 — 2026-05-24

Features

Office-document mining via --mode extract. New mempalace mine <dir> --mode extract ingests PDFs, Word (.docx), PowerPoint (.pptx), Excel (.xlsx), RTF, and EPUB books in addition to the existing source-code/text path. Install with pip install mempalace[extract] — pulls striprtf for RTF and MarkItDown (with [docx,pdf,pptx,xlsx] sub-extras) for the binary formats. Python 3.9 users get RTF coverage only because MarkItDown requires 3.10+. Drawers from the extract path carry extract_mode metadata so the convo miner's "already mined?" check and drawer-id generation stay isolated per mode (#1528). (#1555)
Virtual line numbers + surgical closet pointers. Stored drawers now carry virtual line numbers so the read CLI verb and closet pointers can cite exact line ranges. Closet pointers (Tier 6a) include date+line-range information derived from filename and content-body date parsing (python-dateutil is now a core dep), so MemPalace can point you to exact lines on exact dates rather than just "somewhere in this drawer." (#1555, #1584)
Within-wing hallways. When two entities (people, projects, topics) co-occur in the same drawer, the miner now records a "hallway" — a graph edge connecting them inside that wing. Computed automatically as part of the post-mine step in compute_hallways_for_wing so the graph grows incrementally with new content, no separate command. Foundation for cross-room entity navigation inside one palace. (#1558, #1560)
Cross-wing tunnels promoted from hallways. When the same entity appears in hallways across multiple wings, MemPalace now automatically promotes that into a tunnel — letting queries hop from one person's wing to a project wing they appear in, without anyone calling create_tunnel manually. Topic tunnels from the existing compute_topic_tunnels path remain unchanged. (#1565)
Living-memory dynamics (Hebbian potentiation + Ebbinghaus decay). Hallways and tunnels get stronger every time the same connection is reinforced by new content ("what fires together wires together") and fade gradually if a connection stops appearing in incoming drawers. Navigation weights track real palace usage instead of being static, so retrieval ranking improves over time as the palace is actually used. (#1578)
API-tool transcripts auto-route to wing_api. Conversation transcripts from API-style AI tools (Claude Code, Claude.ai, ChatGPT, Slack-bot exports, generic OpenAI-shape JSON, etc.) now route into a dedicated wing_api instead of mixing into the human-conversation wings. Keeps tool-call traffic from polluting personal/project wings and improves search precision when you're looking for "what did I say" vs. "what did the agent say." (#1236)
Multilingual embedding by default for new installs: embeddinggemma-300m ONNX (q8, MRL→384-dim). MemPalace's previous embedder (all-MiniLM-L6-v2) is trained English-only — cross-lingual cosine similarity on parallel-translated text averages 0.35 across DE/FR/HI/IT/KO/RU (RU at 0.17, near-orthogonal). A Russian-speaking user effectively cannot find their own memories, which breaks the "100% recall" design promise from CLAUDE.md. New EmbeddinggemmaONNX class in mempalace/embedding.py brings this to 0.88 average (validated lossless vs the Ollama gguf via direct ONNX-runtime test). Lazy-downloads onnx-community/embeddinggemma-300m-ONNX (~300 MB) on first use via huggingface_hub. Output is truncated to 384 dims via Matryoshka Representation Learning so the model is a drop-in for ChromaDB's 384-dim collections — no schema change. Sim prefix ("task: sentence similarity | query: ") is applied automatically.

Onboarding (python -m mempalace.onboarding) now offers the multilingual model as the default — choosing it writes embedding_model: embeddinggemma to config.json so subsequent runs pick it up without re-prompting. Existing installs that never set the env var or ran onboarding stay on minilm (back-compat). MEMPALACE_EMBEDDING_MODEL=minilm|embeddinggemma overrides both. Switching models on an existing palace requires re-embedding — run mempalace repair rebuild-index after the change. (#1483)
Multilingual deps moved to core. huggingface_hub, tokenizers, and numpy are now required deps so the multilingual path works out of the box after pip install mempalace. The [multilingual] extra is kept as a no-op alias for back-compat with install scripts. The 300 MB ONNX model itself is still lazy-downloaded on first use, not at install time.
Friendlier ChromaDB EF-name-mismatch error. Switching MEMPALACE_EMBEDDING_MODEL on an existing palace without running rebuild-index previously surfaced ChromaDB's bare Embedding function conflict: new: X vs persisted: Y ValueError — accurate but didn't tell users how to recover. ChromaBackend.get_collection() now wraps that error and points at both options: revert the env var, or run mempalace repair rebuild-index --palace <path>. (#1483)
hooks.auto_save toggle for silent-mode sessions. New config knob (and --silent CLI flag wiring through the save hook) lets users opt out of automatic diary saves on Stop / PreCompact. Useful for "silent mode" sessions where you don't want every conversation captured. Default behavior is unchanged — auto-save still runs unless explicitly disabled. (#711)
Filter common English content words from entity detection. High-frequency English content words ("system", "user", "memory", "project", "context", etc.) were getting tagged as entities by the per-drawer detector and polluting the entity registry as "people." A shipped COCA wordlist (top-N content words) is now consulted during entity classification so these get filtered before they reach the registry. Hardened against malformed JSON in the bundled wordlist. (#1605)
Case-insensitive entity matching at mine time. The initial palace build (mempalace init) matched entity names case-insensitively, but the per-drawer tagger used during incremental mining did not — so the same person was tagged differently between init and ingest ("Aya" vs. "aya" became distinct entities). The incremental tagger now mirrors the case-insensitive matcher, restoring entity-tag consistency across the palace lifecycle. (#1557)

Bug Fixes

Silent data loss in three upsert paths. Three upsert sites (file ingest, conversation ingest, and one repair branch) were calling the embedder on unchunked content, silently truncating at the embedder's max-token limit. Long drawers landed with only their leading section indexed, breaking the "100% recall" promise on long-form content. All three now route through the chunker first so the full document is embedded and stored. (#1540, follow-up to #1539)
Paragraph chunker emitted oversized chunks for long paragraphs. The paragraph splitter assumed paragraphs were always shorter than CHUNK_SIZE and emitted them whole; long paragraphs (legal text, dense technical writeups) silently exceeded the embedder's context window. The splitter now hard-caps each emitted chunk to honor CHUNK_SIZE. (#1538, fixes #1534)
Per-file chunk cap was hardcoded and too low for large transcripts. A safety limit capped chunks per file at a value tuned for source code; mining very large conversation transcripts silently dropped the tail past that cap. Now configurable, with the default raised to cover realistic transcript sizes. (#1554, fixes #1455)
Hook subprocess / ChromaDB deadlock on Windows. Stop/PreCompact hooks could deadlock against an already-open ChromaDB client on Windows, leaving the host (Claude Code) stuck waiting on the hook. Three-part fix: stale-PID timeout on mine-lock reclamation, idle-exit path in the MCP server, and structured errors when the deadlock pattern is detected so the host can recover. (#1562, fixes #1552)
create_tunnel corrupted hyphenated wing names. The endpoint parser split on -, so wings whose name contained a hyphen (mem-palace, my-app) were truncated mid-name and the tunnel pointed at a non-existent endpoint. Endpoint parsing now preserves the full slug. (#1529, fixes #1504)
MCP knowledge-graph cache produced duplicate graphs for symlinked / differently-cased palace paths. Cache key was the raw path string, so /Users/me/.mempalace/palace and /Users/me/.mempalace/Palace (case-folded on macOS) or a symlinked alias produced two separate cached KnowledgeGraph instances pointing at the same SQLite file, with stale-read risk. Cache now normalizes via realpath + normcase so they collapse onto a single canonical key. (#1383, fixes #1372)
Save-hook truncated hyphenated project folder names. Wing-name parser was splitting on - and keeping only the first segment, so mem-palace became mem. Fix preserves the full project-folder slug so hyphenated palaces stay coherent across hook invocations. (#1424, fixes #1410)
Miner silently skipped symlinks. Users were confused about missing data after mining; the miner was skipping symlinks without surfacing it. Now logs each skipped symlink with the reason so the gap is visible. (#1466, fixes #1462)
Host-leaked PYTHONPATH could shadow MemPalace's own modules at import. Package __init__ now strips leaked entries on import so the imported MemPalace is always the installed one. (#1439, fixes #1423)
macOS stock-bash hook scripts. Hook scripts used mapfile (bash 4+), breaking macOS's stock /bin/bash 3.2. Switched to a sed pipeline so hooks work out of the box on every Mac. (#1441, fixes #1440)
Plugin Stop/PreCompact hooks could hang indefinitely on a stuck child. Bounded timeout ensures the host can always make forward progress even if MemPalace's child process is unhealthy. (#1470, fixes #1465)
MCP handlers now return structured JSON-RPC errors for malformed input. Unknown parameter names returned -32602 Invalid params instead of an unstructured Python TypeError; parameters of the wrong shape return a structured error instead of a raw traceback. (#1500, #1513)
CLI distinguished "palace doesn't exist" from "palace exists but is empty". Two states that look the same to a new user are now reported separately with actionable next-step messages for each. (#1532, fixes #1498)
mempalace repair post-pass: VACUUM + FTS5 rebuild. After a palace repair, the SQLite knowledge-graph file kept fragmented pages and a stale FTS5 index; running VACUUM and rebuilding FTS5 at the end reclaims disk and restores search performance. (#1523)
Convo miner mode isolation. The "already mined?" check and drawer-id generation ignored extract_mode, so switching modes either re-mined the same content (data dup) or collided drawer IDs across modes. Scoping by mode keeps each mode's drawers isolated and dedup-correct. (#1528, fixes #1505)
FTS5 validation at end of mine. Mining now validates the FTS5 index integrity as a post-step so corruption is caught at write time, not at read time. (#1548, fixes #1537)
hooks_cli crashed on shallow install paths. Code indexed Path.parents[3] assuming a deep install tree, raising IndexError when MemPalace ran from a shallow path (e.g. /opt/mp). Adds a guard so shallow installs no longer crash on hook commands. (#1585)
HNSW segment quarantine: zero-byte vs missing-dim. Earlier quarantine heuristic flagged any HNSW segment missing the dim metadata field as corrupt, but most were recoverable; the check now distinguishes recoverable-missing-dim from actually-corrupt, preserving working index segments. Zero-byte link-list files (partially-written segments) are now rejected outright. (#1452, #1461, fixes #1457)
Mine-lock holder file written as UTF-8 instead of cp1252. Non-ASCII Windows usernames and paths no longer corrupt the lock file and break stale-lock detection. (#1438)
Miner slot claim now writes a placeholder PID immediately. Crash between claim and PID-write no longer leaves a phantom lock. (#1543, fixes #1443)
mine_convos now runs inside mine_palace_lock. Two concurrent convos mine invocations can no longer corrupt the index. (#1477)
Migration tool cleanup. ChromaDB-version migration tool now closes its SQLite connection and removes the temp palace directory if an exception fires mid-migration; failed migrations stop leaking file handles and disk. Entity-registry atomic-write now deletes its .tmp sidecar if the write or rename fails. (#1216, #1408, fixes #1373)
Repair tool tolerated empty/None metadata cells. ChromaDB occasionally returns cells with empty metadata dicts or None during rebuild; both are now coerced to sensible defaults so the rebuild completes and otherwise-stuck palaces recover. (#1459, #1445, fixes #1426)
create_tunnel MCP handler now propagates errors. Bad endpoint or direction was being swallowed and returned as misleading success; now propagates as a structured MCP error. (#1546, fixes #1473)
Explicit tunnels were stored at a hardcoded ~/.mempalace/tunnels.json path that ignored MempalaceConfig.palace_path. Drawers, KG triples, the people map, and every other piece of palace state honour the configured palace_path (and the MEMPALACE_PALACE_PATH env var), but palace_graph._TUNNEL_FILE was a module-level constant initialised once from os.path.expanduser("~") + "/.mempalace/tunnels.json". Under any setup where $HOME is isolated from the configured palace — subagent profiles with their own $HOME, sandboxes, multi-tenant hosts, container mounts that move the palace to /srv/ — drawers landed in the configured palace while tunnels silently landed in a different file that no other process touching the same palace could see. Worst case is the agentic one: an isolated worker calls create_tunnel then list_tunnels and gets back its own write from the bubble, so the worker self-confirms a tunnel that doesn't exist in the shared palace and reports completion to the orchestrator. palace_graph._TUNNEL_FILE is replaced by _get_tunnel_file() which derives the path from a new MempalaceConfig.tunnel_file property (sibling of palace_path). The default single-user install is unchanged because the default palace_path is still ~/.mempalace/palace and its sibling tunnels.json is the legacy path. Backwards-compatibility: if the configured tunnel file does not exist but a file is present at the pre-3.3.6 hardcoded ~/.mempalace/tunnels.json path AND the two paths differ, _load_tunnels logs a one-line WARNING naming both paths and returns an empty list — we intentionally do NOT auto-migrate because silently merging tunnel state across two locations risks clobbering newer data; the user moves or copies the file themselves. (#1467)
create_tunnel did not validate that the source and target rooms actually exist in the chroma index. _require_name only checked that wing/room names were non-empty strings; nothing queried the collection to confirm at least one drawer carried matching {wing, room} metadata. Pointing an explicit tunnel at a phantom room — common when an agent fabricates a room name it expects to exist, or types a slug wrong — silently succeeded. Combined with the read-bubble described in the previous fix, an agent could create_tunnel → list_tunnels and have both calls return its own bogus write. create_tunnel now calls _check_room_exists(wing, room, col) for both endpoints before persisting an explicit tunnel; if either endpoint has zero matching drawers the call raises ValueError naming the offending wing/room pair. Three deliberate carve-outs: (1) kind != "explicit" skips validation because topic tunnels generated by compute_topic_tunnels use synthetic topic:<name> room identifiers that don't correspond to real chroma rooms; (2) _get_collection returning None (palace not yet created, transient backend failure, tests without a real chroma backend) skips validation rather than fail-closed — matches the tolerance pattern used everywhere else in palace_graph; (3) exceptions raised by the underlying col.get(where=..., limit=1, include=[]) query are logged and treated as "can't verify, allow" so a temporary index fault never blocks legitimate writes. Behaviour change: existing callers that previously created tunnels pointing at empty rooms (e.g. as scaffolding before mining them) will now raise ValueError. File the drawer first, then create the tunnel — this is the order the documentation has always recommended. (#1468)

Performance

Convo miner pre-fetches mined-set once. Was issuing one WHERE query per file to check "already mined?"; now pre-fetches the full mined set once, slashing wall time on large transcript corpuses. (#1474)
rebuild_index progress callback. Multi-hour rebuilds now report progress with default ETA printer; users no longer have to guess whether the process is making progress. (#1487)
MCP cold-start diagnostics + opt-in warmup. Adds visibility into which embedder is loading and how long it takes, plus an opt-in warmup path so users can see and address slow first-query latency. (#1530, fixes #1495)

Internal

palace_graph._TUNNEL_FILE (module-level constant) replaced by _get_tunnel_file(config=None) and _legacy_tunnel_file(). Tests previously monkeypatching the constant must now monkeypatch the resolver functions. The tests/test_palace_graph_tunnels.py::_use_tmp_tunnel_file helper, tests/test_closets.py::TestTunnels setup/teardown, and three tests in tests/test_miner.py were updated accordingly. Topic-tunnel tests in test_miner continue to work without stubbing _get_collection because kind="topic" short-circuits the new validation path.

3.3.5 — 2026-05-09

Bug Fixes

MCP tool_search now retries once on transient Error finding id from chromadb's HNSW flush window. After a bulk CLI mine, ChromaDB's HNSW segment metadata can be unflushed for ~30-60s; wing-scoped MCP search hits Internal error: Error finding id during that window. tool_search now detects this transient via response-shape sniffing, drops both the MCP-local client cache and _DEFAULT_BACKEND._clients / _freshness for the palace, sleeps 2s, and retries once. Successful retries are tagged with index_recovered: true so callers can observe when it fired; non-transient errors bypass the retry path entirely. Partial fix for the broader #1315 cluster — tool_check_duplicate and other index-touching tools still need the same wrapper. (#1396, refs #1082, #1315)
mempalace_diary_read silently dropped entries on agent-name case mismatch. tool_diary_write stored the agent metadata verbatim after sanitize_name, which preserves case, while tool_diary_read filtered by exact match. Writing as "Claude" and reading as "claude" (or vice-versa) returned zero rows. Both endpoints now lowercase agent_name immediately after sanitization, so reads are case-insensitive and the default per-agent wing slug is stable across casings. Behavior change: entries written prior to this fix under mixed-case agent names will not match the new lowercase filter; run mempalace repair if you need to migrate legacy diary metadata. (#1243)
Knowledge-graph triples with valid_to < valid_from were silently invisible. KnowledgeGraph.query_entity() filters with valid_from <= as_of AND valid_to >= as_of, so an inverted interval matches no as_of and the row is durably stored but unreachable — a P0 data-integrity foot-gun any caller that mixes up the two date params can hit. add_triple() now rejects inverted intervals at write time with a clear ValueError naming both bounds. Open intervals (one bound only) and point-in-time facts (valid_from == valid_to) remain accepted unchanged. (#1214)
ChromaBackend.close_palace() / close() did not release the SQLite file lock. Evicted clients sat in _clients without close(), and chromadb 1.5.x retains the rust-side SQLite lock until GC. Reopening the same palace path after shutil.rmtree + recreate within one process failed with SQLITE_READONLY_DBMOVED (code 1032). New _close_client() helper now calls PersistentClient.close() (with a try/except fallback for older chromadb) on close_palace(), on whole-backend close(), and on the _client() invalidation path that detects a missing chroma.sqlite3. The mtime/inode auto-invalidation branch is intentionally left alone — callers there may still hold a live ChromaCollection. (#1067, #1105)
EntityRegistry.save() could leave a corrupt or empty entity_registry.json on crash. Path.write_text() is not atomic — kernel sees open('w') (truncate), write, close, and any failure between truncate and full-flush (power loss, OOM, FS-full, kill -9) wipes the months-of-mining people/projects map silently (the registry's load() swallows JSONDecodeError). Save now writes to a sibling .tmp in the same directory, fsyncs, chmod 0o600s, then os.replace()s into place — atomic on POSIX and Windows. The previous registry stays intact on any crash before the rename returns. (#1215)
miner.detect_room bidirectional substring matching caused systemic misrouting. The priority-1 (path parts) and priority-2 (filename) checks used c in part or part in c against room names + keywords, so any token that was an unbounded substring of a room name (or vice versa) matched. Priority-1 iterates left-to-right and returns on first match, so views/billing-page/src/Foo.test.tsx routed to an interviews room because "views" in "interviews" matched before reaching billing-page. Both call sites now use a _name_matches helper that compares names as equal or as separator-bounded tokens of each other (split on -, _, ., /). (#1004, closes #1002)
mempalace compress crashed on large palaces. regenerate_closets fetched all closet_llm drawers in a single col.get(), which trips SQLITE_MAX_VARIABLE_NUMBER on palaces above ~32k drawers. Mirrors the #851 fix in miner.py: drawer fetch is now paginated at batch_size=5000. Per-source aggregation works across batches, so the LLM regeneration call still groups chunks correctly. (#1073, #1107)
CLI and fact_checker --stdin mojibaked non-ASCII content on Windows. Python defaults sys.stdin/stdout/stderr to the system ANSI codepage (cp1252/cp1251/cp950), so mempalace search > out.txt and piped fact_checker invocations corrupted Cyrillic / CJK drawer text at the process boundary. New mempalace/_stdio.py helper reconfigures all three streams to UTF-8 on sys.platform == "win32", with per-stream errors policy: surrogateescape on stdin (preserves bad bytes from redirected files for the consumer's parser), replace on stdout/stderr (substitutes U+FFFD instead of UnicodeEncodeError-ing mid-print). With this, all three user-facing console_scripts (mcp_server, hooks_cli, cli/fact_checker) now reconfigure identically on Windows. (#1282)
MCP knowledge-graph tools forwarded malformed date strings to SQLite. tool_kg_query (as_of), tool_kg_add (valid_from), and tool_kg_invalidate (ended) accepted any string and produced empty result sets on natural-language inputs like "March 2026" or "yesterday" — callers (especially LLM agents) could not distinguish "no fact at this time" from "your date format was unrecognized." New sanitize_iso_temporal() validator in config.py (with sanitize_iso_date() retained as a backwards-compat wrapper) accepts YYYY-MM-DD, YYYY-MM-DDTHH:MM:SSZ, and YYYY-MM-DDTHH:MM:SS+00:00 (normalized to the Z form), and passes None/"" through unchanged; all three KG tools call it before values reach the storage layer. Partial dates (YYYY, YYYY-MM), naive datetimes, and non-UTC timezone offsets are rejected because KG queries compare TEXT temporal values where mixed formats silently return wrong results. Behavior change: previously-silent date typos now raise a clear ValueError naming the offending field; partial-date inputs that worked in 3.3.4 ("2026", "2026-05") no longer parse — pass a full YYYY-MM-DD or a canonical UTC datetime instead. (#1164, #1167, #1374, #1417)
MCP server's _kg was a module-level singleton. Multi-tenant hosts that rotate MEMPALACE_PALACE_PATH between tool calls hit the wrong sqlite file, because the KG was constructed once at import time while the ChromaDB side was already per-call via _get_client(). The KG is now resolved per-call through a lazy per-path cache (_kg_by_path keyed by os.path.abspath, with a double-checked-locking init under _kg_cache_lock). tool_reconnect drains and close()s cached KGs alongside the existing chroma reconnect. A _call_kg retry guard catches sqlite3.ProgrammingError once after a reconnect race. (#1136, #1160)
mempalace repair can now recover palaces whose HNSW segment writer is stuck on apply_logs. Both the existing --mode legacy rebuild and the inline cli.cmd_repair path call Collection.count() as their first read — exactly the call that raises chromadb.errors.InternalError: Failed to apply logs to the hnsw segment writer on the corruption class introduced upstream and reported in #1308. Repair would print Cannot recover — palace may need to be re-mined from source files even though the underlying SQLite tables were fully intact (the corruption lives in the on-disk index files, not the data layer). New --mode from-sqlite reads (id, document, metadata) rows directly from chroma.sqlite3 via a segments → embeddings → embedding_metadata join, never opens a chromadb client against the corrupt palace, and re-upserts everything into a fresh palace at --palace. --source PATH extracts from a corrupt palace already moved aside; --archive-existing handles the in-place case by renaming the existing palace to <palace>.pre-rebuild-<timestamp> before reading from it. Documents are re-embedded under the user's configured embedding function (the original HNSW vectors live in the corrupt data_level0.bin and cannot be recovered, but the embedding model is deterministic so search results remain semantically equivalent). Verified end-to-end on a 52,300-row real-world corrupt palace. (#1308)

Documentation

CONTRIBUTING.md git-identity guidance. New section asks contributors to verify git config user.name and git config user.email before pushing, with an explicit warning for agentic coding tools that may not inherit the user's normal Git config. Avoids placeholder/template author values in commit history. (#1385, closes #1317)

Internal

Test reliability: multiprocessing start method. tests/test_palace_locks.py and tests/test_chroma_collection_lock.py switched from fork to spawn for child processes. Under Python 3.13 the pytest parent is multi-threaded by the time these tests run (chromadb + onnxruntime each spawn background threads on import); fork snapshotting that state into the child without the threads themselves deadlocked Linux 3.13 and macOS CI jobs indefinitely while Linux 3.9 / 3.11 / Windows finished normally. macOS additionally forbids fork-without-exec via CoreFoundation. spawn re-imports modules in the child (~0.5s per Process — bounded by the 10 subprocesses these tests fork) but is safe under threading. (#1431)
Test cleanup: SQLite connection lifecycle. Wrapped naked conn = sqlite3.connect(...) blocks in tests/test_backends.py, tests/test_sources.py, and tests/test_repair.py with contextlib.closing(...). The flat conn.close() pattern at the end of each test leaked the connection on any exception or assertion failure between connect and close, producing ResourceWarning: unclosed database noise in CI logs and creating a secondary risk of advisory-lock starvation on Python 3.13 / macOS. Mirrors the try/finally pattern already used in production code. (#1430)

3.3.4 — 2026-04-30

Added

mempalace init now prompts to mine the same directory. After entity confirmation, room detection, and gitignore guard, init shows a one-line scope estimate (e.g. ~423 files (~12 MB) would be mined into this palace.) computed from its existing corpus walk, then asks Mine this directory now? [Y/n] (default yes) and runs mine() in-process if accepted. The estimate fires before the prompt so users on a real corpus aren't surprised by a minutes-long ChromaDB write. Declining prints the exact mempalace mine <dir> command for later. (#1181)
New --auto-mine flag on mempalace init for the non-interactive path (mempalace init --auto-mine <dir> skips the mine prompt and runs mine directly). --yes retains its existing scope of entity auto-accept only and still prompts for the mine step, so existing scripted callers see no behaviour change; combining --yes --auto-mine gives a fully non-interactive setup. (#1181)
Cross-wing topic tunnels. When two wings have confirmed TOPIC labels in common (the LLM-refine bucket from mempalace init --llm), the miner now drops a symmetric tunnel between them at mine time so the palace graph reflects shared themes (frameworks, vendors, recurring concepts). Tunnels are routed through the existing create_tunnel storage so they share dedup and persistence with explicit tunnels. Topic tunnels are stored under a synthetic topic:<name> room and tagged with kind: "topic" on the stored dict — this keeps them distinct from literal folder-derived rooms of the same name (a wing with both an Angular folder room and an Angular topic tunnel no longer collides at follow_tunnels read time) and gives LLMs scanning list_tunnels a visible discriminator. Threshold is configurable via MEMPALACE_TOPIC_TUNNEL_MIN_COUNT env var or topic_tunnel_min_count in ~/.mempalace/config.json (default 1). Manifest-dependency overlap and per-topic allow/deny lists remain out of scope. (#1180)
Context-aware corpus detection at mempalace init. A new Pass 0 runs at the start of init — before entity detection — and answers one question: is this corpus an AI-dialogue record, and if so, which platform and what persona names has the user assigned to the agents? Tier 1 is a free regex heuristic (well-known AI brand terms + turn-marker patterns, with a co-occurrence rule that suppresses ambiguous terms like Claude/Gemini/Haiku when no unambiguous AI signal is present, so French novels and astrology forums don't false-positive). Tier 2 is an LLM call (~$0.01 with Anthropic Haiku, free with local Ollama/LM Studio/llama.cpp/vLLM) that extracts user_name and agent_persona_names from dialogue structure. Result is persisted to <palace>/.mempalace/origin.json with a schema_version: 1 envelope so downstream tools can read it. Entity classification then routes names matching agent_persona_names (case-insensitive) into a new agent_personas bucket instead of people, so a Claude Code transcript no longer misclassifies the user's Echo/Sparrow/Cipher agents as biological people. llm_refine receives the same context as a system-prompt preamble so it can disambiguate other ambiguous candidates with corpus-level knowledge too. Backwards compatible: callers that don't pass corpus_origin see the v3.3.3 return shape unchanged. (#TBD)
mempalace init runs LLM-assisted refinement by default. v3.3.3 made --llm opt-in; the LLM-assisted path is qualitatively better (extracts persona names, refines ambiguous classifications) so it now runs by default. Provider precedence is unchanged — Ollama at http://localhost:11434 first, then openai-compat, then anthropic with API key. Never blocks init on a missing LLM: if no provider is reachable (Ollama not running, no API key set), init prints a one-line message pointing at --no-llm and falls through to the heuristic-only path. --no-llm is the new explicit opt-out. The legacy --llm flag is preserved as a deprecated alias of the default so scripted callers see no behaviour change. Cost story: zero for users with a local LLM (the majority on this repo), ~$0.01 per init for users with ANTHROPIC_API_KEY set who explicitly choose --llm-provider anthropic, zero for users with no LLM (graceful fallback). (#TBD)
mempalace mine --redetect-origin flag. Re-runs corpus-origin detection on the current corpus state and overwrites <palace>/.mempalace/origin.json. Useful when the corpus has grown since mempalace init and the stored origin may be stale. Heuristic-only by design (the flag is meant to be cheap); re-run mempalace init for full Tier 2 LLM refinement. Default mempalace mine does not touch origin.json — the flag is opt-in. (#TBD)

Bug Fixes

MCP server tool_diary_write SIGSEGV when default EF provider differs. mcp_server._get_collection bypassed ChromaBackend.get_collection and called client.get_collection / client.create_collection without embedding_function=. ChromaDB 1.x persists the EF identity (its name()) with the collection but not the EF instance/configuration, so the MCP server's reopen silently bound chromadb's built-in DefaultEmbeddingFunction — its name() matches mempalace.embedding's spoofed "default" so the identity check passes, but its provider list is chromadb's default rather than the user's resolved device. The miner / Stop hook ingest path routes through the backend helper and binds the configured EF instead. On bleeding-edge interpreters (python 3.14 + chromadb 1.5.x on Apple Silicon) the default provider selection could SIGSEGV the host process on first col.add(), killing the MCP stdio server and leaving every subsequent tool call returning Connection closed until Claude Code was relaunched. _get_collection now reuses ChromaBackend._resolve_embedding_function() on the reopen branches that actually open a collection (warm-cache reads stay zero-cost), matching the miner/backend path. (#1299, follow-up to #1262 / #1289)
Hooks no longer recreate ~/.mempalace/ after the user removes it. When ~/.mempalace/ is deleted (a strong "do not auto-capture" signal), the next Stop, PreCompact, or SessionStart hook would silently rebuild the dir hierarchy and ingest existing transcripts: _log() called STATE_DIR.mkdir(parents=True, exist_ok=True) unconditionally, so the very act of writing [HH:MM] SESSION START … recreated ~/.mempalace/hook_state/; subsequent calls in the save path then materialized palace/, wal/, knowledge_graph.sqlite3, and N drawers from ~/.claude/projects/*.jsonl. All four entry points (hook_stop, hook_precompact, hook_session_start, and _log itself) now check a new module-level PALACE_ROOT = Path.home() / ".mempalace" constant first and short-circuit (returning {} on stdout, never logging) when the directory is absent. The user-removable directory becomes a kill-switch — rm -rf ~/.mempalace is now a stable state. Net: 23 lines added in mempalace/hooks_cli.py, 5 unit tests in tests/test_hooks_cli.py. (#1305)
Cross-wing topic tunnels for hyphenated dir names. mempalace init recorded the topics_by_wing registry key under the raw directory name (e.g. mempalace-public), while mempalace.yaml's wing field used the lower-cased + separator-collapsed slug (mempalace_public). At mine time the miner read the slug from the yaml and missed the registry, so _compute_topic_tunnels_for_wing returned 0 silently. Real-world: any project whose folder contained a hyphen or space lost every topic tunnel. Now both call sites route through a shared normalize_wing_name() in config.py. (#1194, follow-up to #1180)
CLI mempalace search retrieval quality. The CLI was using pure ChromaDB cosine distance with no BM25 rerank, so drawers containing every query term but embedding as noise (directory listings, diff output, shell logs) scored Match: 0.0 alongside genuinely irrelevant results with no way to tell them apart. Wired the CLI through the same _hybrid_rank the mempalace_search MCP tool already used, and surfaced both cosine= and bm25= scores in the output so users see which component of the match is firing. MCP search was unaffected; this fixes the human-facing CLI parity gap.
Legacy-palace distance-metric warning. CLI search now detects palaces created before hnsw:space=cosine was consistently set and prints a one-line notice pointing at mempalace repair. Without the warning such palaces silently used L2 distance, under which the similarity display floored every result to Match: 0.0. New palaces mined today already set cosine correctly and now have invariant tests pinning that behavior so future refactors can't silently regress it. (#1179)
Graceful Ctrl-C during mempalace mine. Interrupting a long mine no longer dumps a multi-frame KeyboardInterrupt traceback. The main file-processing loop now catches the signal, prints files_processed: N/M, drawers_filed: K, and last_file: so the user knows what landed, then exits with code 130 (standard SIGINT). Already-filed drawers are upserted idempotently on re-mine via deterministic IDs, so resuming is safe. The hooks PID lock at ~/.mempalace/hook_state/mine.pid is now also actively cleaned up in a finally when its entry points at us — clean exit, error, or interrupt — preventing the next hook fire from briefly waiting on a stale PID. (#1182)
mempalace init is now idempotent across re-runs. Running init twice on the same project produced different origin.json results because the first run wrote entities.json into the project directory, and the second run's corpus-origin sampling included that file as corpus content — shifting Tier 1's character-density math. Sampling now skips the per-project artifacts (entities.json, mempalace.yaml), so re-running init produces the same classification it did the first time. Pinned by an integration test in tests/test_corpus_origin_integration.py. (#TBD)

3.3.3 — 2026-04-23

Bug Fixes

Install regression — mempalace-mcp console script is now declared in pyproject.toml alongside .claude-plugin/plugin.json's reference to it. In v3.3.2 the two drifted apart (plugin.json shipped the new "command": "mempalace-mcp" form before the matching entry point landed), so every fresh pip install mempalace==3.3.2 produced a Claude Code plugin config pointing at a binary that wasn't installed. (#1093, #340)
Restore silent-save visibility after the Claude Code 2.1.114 client regression — production transcript saves were failing silently until this PR. (#1021)
Paginate status-path metadata fetches so large palaces don't trip SQLite variable limits. (#851)
Resolve the Claude plugin hook runner across platform / plugin-dir variations; previously broke on Windows and some macOS layouts. (#942)
Real python3 resolution for .sh hooks with a MEMPAL_PYTHON override path. (#833)
Add optional wing parameter to tool_diary_write / tool_diary_read and derive per-project wing from the Claude Code transcript path when writing from the stop hook — diary entries from different projects no longer collapse into a shared default wing. (#659)
Treat empty string as "no filter" in mempalace_search wing/room; LLM agents that default to filling every optional parameter with "" no longer get bounced with must be a non-empty string. (#1097, #1084)
Broaden _wing_from_transcript_path to handle Claude Code project folders without a -Projects- segment (e.g. ~/dev/<parent>/<project>, ~/code/<project>). The project name is now derived from the final dash-separated token of the encoded folder, so Linux users with code outside ~/Projects/ get per-project diary scoping instead of falling through to wing_sessions. (#1145, follow-up to #659)
mempalace_diary_read(wing="") now returns diary entries from every wing this agent has written to, matching the #1097 "empty-string as no filter" pattern. Previously defaulted to wing_<agent>, siloing entries that hooks wrote to project-derived wings. (#1145)
mempalace mine now skips the generated entities.json file so its contents aren't re-ingested as project content. (#1175)

Improvements

Deterministic hook saves. Save hook now uses a silent Python API path, so successive hook invocations produce reproducible results and zero data loss on the hot path. (#673)
Graph cache with write-invalidation inside build_graph() — warm-path calls no longer rebuild the palace-graph per request. (#661)
mempalace init entity detection overhaul. Canonical project names now come from package manifests (package.json, pyproject.toml, Cargo.toml, go.mod) and real people come from git commit authors, rather than being inferred from prose. Includes union-find dedup across name/email aliases, bot filtering that keeps @users.noreply.github.com humans, and automatic "mine" flagging by contribution share. (#1148)
Regex detector accuracy. CamelCase extraction so MemPalace, ChromaDB, OpenAI aren't fragmented; tighter versioned/hyphenated pattern kills context-manager / multi-word false positives; dialogue ^NAME:\s requires ≥2 hits so Created: <date> metadata stops classifying field names as people; expanded stopwords for common English participles and descriptors; high-pronoun signal classifies as person rather than dumping to uncertain. (#1148)
Init → miner wire-up. Confirmed entities merge into ~/.mempalace/known_entities.json on init, which the miner reads to tag drawer metadata for entity-filtered search. Previously init's output was not consumed by the miner; the per-project entities.json is kept as an audit trail. (#1157)
Case-insensitive project dedup across manifest, git, and convo sources so casing variants of the same project name collapse into one review entry. (#1175)

Added

i18n: Belarusian translation. (#1051)
i18n: entity detection for German, Spanish, and French locales. (#1001)
i18n: Traditional + Simplified Chinese entity detection. (#945)
mempalace init --llm: optional LLM-assisted entity classification. Defaults to local Ollama (zero-API); also supports any OpenAI-compatible endpoint (LM Studio, llama.cpp server, vLLM, OpenRouter, etc.) and the Anthropic Messages API. Runs interactively with a progress indicator; Ctrl-C cancels cleanly and returns partial results. Useful for prose-heavy folders where the regex detector struggles (diaries, transcripts, research notes). Opt-in only — default init path remains zero-API. (#1150)
Claude Code conversation scanner. ~/.claude/projects/<slug>/ directories now contribute project entities using each session's authoritative cwd metadata, avoiding slug-decoding ambiguity. (#1150)

Known — deferred to v3.3.4

HNSW parallel-insert SIGSEGV when hnsw:num_threads is unset on collection creation (#974) — fix in-flight as #976, awaiting rebase against develop.

3.3.2 — 2026-04-19

Bug Fixes

Fix silent drop of .jsonl files in project miner; raise MAX_FILE_SIZE cap from 10 MB to 500 MB so large transcripts no longer fall through unnoticed. Adds a tandem sweeper — a message-level, timestamp-coordinated, idempotent safety net that catches anything the primary miner missed. (#998)
mempalace sweep <target> CLI to run the sweeper on demand against a transcript file or a directory. (#998)
Guard Layer3.search_raw against None doc/meta rows returned by ChromaDB — prevents AttributeError crashes on mixed-schema palaces. (#1011, #1013)
Guard searcher API path, closet loop, and miner status histogram against None metadata; matching guards added to tool_status / list_wings / list_rooms / get_taxonomy in the MCP server. (#999)
Upgrade chromadb floor to >=1.5.4 for Python 3.13 / 3.14 compatibility and pin upper bound to <2 so future breaking majors don't silently install. (#1010)
Fix Unicode checkmark rendering on Windows terminals that can't encode the ✓ glyph — avoids UnicodeEncodeError crashes on first-run output. (#681)
quarantine_stale_hnsw — on open, detect HNSW segment directories whose data_level0.bin is significantly older than chroma.sqlite3 and rename them out of the way. Recovers cleanly from HNSW/sqlite drift that otherwise causes SIGSEGV on count() / query(...) (the chroma-core/chroma#2594 failure mode). Rebuilds the index lazily on next use. (#1000)
PID file guard — mine writes a per-source-directory PID file and refuses to start if an existing mine is still running, preventing process stacking that bloats HNSW and wedges concurrent writes. Includes cross-platform PID liveness check (os.kill(pid, 0) terminates on Windows, so the guard falls back to a platform-aware probe). (#1023)

Improvements

RFC 001 §10 — typed backend contracts. BaseBackend now returns typed QueryResult / GetResult dataclasses and PalaceRef for palace identity; registry-based backend discovery. Internal refactor; no user-facing API change. (#995)
RFC 002 §9 — source adapter scaffolding. Introduces BaseSourceAdapter, adapter registry, and PalaceContext — the plumbing that future pluggable ingest sources will target. Internal refactor; no user-facing API change yet. (#1014)

Documentation

RFC 002 — full specification for the source adapter plugin system (future pluggable ingest). (#990)
First-run help text and README now reference the real ~/.claude/projects/<project>/ path shape instead of the placeholder /path/to/transcripts. (#996, #1012)

Internal

Harden sweeper for production: verbatim tool blocks, full session_id, logged failures.
Address Copilot review on #995: cursor tie-break, honest metrics, accurate comments.
Test hygiene: avoid ONNX network download in update-length validation tests; dedup update-length-validation tests; fix Windows file-lock in cache-invalidation test.

3.3.1 — 2026-04-16

New Features

Multi-language entity detection — lexical patterns (person verbs, pronouns, dialogue markers, project verbs, stopwords, candidate character classes) now live in the optional entity section of each locale JSON under mempalace/i18n/<lang>.json. Every public function in entity_detector accepts a languages= tuple and unions patterns across enabled locales. Default stays ("en",) so existing English-only callers are unchanged. (#911)

Five new fully-supported locales with CLI strings, AAAK compression instructions, and entity-detection patterns:
- Brazilian Portuguese pt-br (#156)
- Russian ru (#760)
- Italian it (#907)
- Hindi hi (#773)
- Indonesian id (#778)
MempalaceConfig.entity_languages — persistent palace-level language selection; MEMPALACE_ENTITY_LANGUAGES env override; mempalace init --lang en,pt-br flag that saves to ~/.mempalace/config.json (#911)
Per-language candidate_pattern — non-Latin scripts register their own character class, so names like João, Инна, राज are no longer silently dropped by the ASCII-only default (#911)
VSCode devcontainer matching the CI environment (#881)
MEMPAL_VERBOSE env toggle — developers see diaries surfaced in chat while the default remains silent (#871)
created_at timestamps included in search results (#846)

Bug Fixes

i18n / Unicode

Script-aware word boundaries for combining-mark scripts — Python's \b fails on Devanagari vowel signs (ा ी ु), Arabic, Hebrew, Thai, Tamil, Khmer etc., truncating names like अनीता → अनीत and making person-verb patterns never fire. Locales now declare an optional boundary_chars field and the i18n loader expands \b into a script-aware lookaround boundary (#932)
Case-insensitive BCP 47 language code resolution — --lang PT-BR, zh-cn, Pt-Br previously fell through to English silently; now resolve to the canonical locale file via lowercase matching, with the entity-pattern cache keyed on the canonical form so casing variations share one cache entry (#928)
Wire i18n candidate patterns into miner._extract_entities_for_metadata(), palace.build_closet_lines(), and entity_registry.extract_unknown_candidates() — three code paths that still hardcoded ASCII-only [A-Z][a-z]{2,} and silently missed Cyrillic, accented Latin, and non-Latin entity metadata tags (#931)
Explicit encoding="utf-8" on Path.read_text() calls across entity_registry, instructions_cli, split_mega_files, and onboarding tests — prevents Windows GBK (and other non-UTF-8) locales from corrupting UTF-8 files (#946, #776)
ko.json status_drawers used {drawers} instead of {count}, showing the raw template string instead of the number (#758)
Move test_i18n.py from inside the installed package into tests/ so pytest actually collects it; remove the sys.path.insert hack (#758)
Dialect.from_config() defaulted to current_lang() (module-global) when config had no lang key — replaced with explicit "en" fallback for determinism (#758)

Other

Guard KnowledgeGraph.close() and query_relationship/timeline/stats methods with the instance lock to prevent concurrent-access corruption (#887, #884)
Replace invalid {"decision": "allow"} with {} in hook responses — the string wasn't a valid decision value and triggered schema warnings (#885)
entity_registry.research() defaults to local-only — previously made outbound Wikipedia HTTPS requests without explicit user opt-in; callers now must pass allow_network=True (#811)
Precompact hook no longer blocks compaction when it fails or takes too long (#856, #858, #863)
Redirect stdout to stderr during MCP server import so library logging can't corrupt the JSON-RPC channel (#225, #864)
mempalace init auto-adds per-project files to .gitignore in git repositories so users don't accidentally commit mempalace.yaml / entities.json (#185, #866)
Searcher guards against empty ChromaDB query results that previously raised on edge-case corpora (#195, #865)
Return empty status instead of an error on a cold-start palace with no drawers yet (#830, #831)
Restrict file permissions on sensitive palace data (#814)
Slack transcript importer writes a provenance header and preserves speaker IDs (#815)
Allow mempalace mine to run in directories without a local mempalace.yaml and surface the missing-yaml warning on stderr (#604)
Security hook injection fix (#812)
Save hook auto-mines transcripts even when MEMPAL_DIR is unset (#840)
Pin the Pages custom domain via a shipped CNAME in the deploy artifact (#877)
Version drift safeguard — sync pyproject + version.py + README badge in one place (#876)
Deploy docs workflow now runs on develop only, preventing accidental main-branch deploys (#845)

Improvements

Regex compilation optimization for entity extraction — pre-compile per-entity pattern sets once and cache by (name, languages) tuple, so multi-language callers don't thrash the cache (#880)
Knowledge-graph value sanitization now preserves natural punctuation (commas, colons, parentheses) that commonly appears in KG subject/object values (#873)

Documentation

Clarify that mempalace init requires a <dir> argument in CLI help text (#210, #862)
Domain name and specific impostor sites called out in the scam-alert section (#869)
Tightened SECURITY.md with a real version-support policy and the GHPVR-only reporting channel (#810)
Fixed stale pyproject.toml URLs (#853)
v4 planning prep (#852)

Internal

palace_graph tunnel helper test coverage (#908)

3.3.0 — 2026-04-13

New Features

Closet layer — a compact searchable index of pointers to verbatim drawers, enabling fast topical lookup without reading all content (#788)
BM25 hybrid search — closets boost ranking, drawers remain the source of truth (#795, #829)
Entity metadata on every drawer for filterable search (#829)
Diary ingest — day-based rooms for conversation transcripts (#829)
Cross-wing tunnels — explicit links between rooms in different wings for multi-project agents (#829)
Drawer-grep — returns the best-matching chunk plus adjacent context drawers (#829)
Offline fact checker against the entity registry and knowledge graph (#829)
LLM-based closet regeneration — optional, bring-your-own endpoint, no mandatory API key (#793)
Hall detection — routes drawer content to emotions / technical / family / memory / identity / consciousness / creative halls, enabling hall-based graph connectivity within wings (#835)

Bug Fixes

Repair max_seq_id corruption caused by _fix_blob_seq_ids misinterpreting chromadb 1.5.x's sysdb-10 BLOB format (b'\x11\x11' + ASCII digits) as legacy 0.6.x big-endian BLOBs. The shim now skips the max_seq_id table entirely and guards the embeddings branch with a prefix check. New subcommand mempalace repair --mode max-seq-id [--from-sidecar <path>] restores affected palaces. Fixes silent drawer-write drops that began after chromadb 1.5.x upgrades on palaces that still had BLOB-typed max_seq_id rows at migration time.
Set hnsw:space=cosine metadata on all collection creation sites — fixes broken similarity scoring under ChromaDB's default L2 distance (#807, #218)
File-level locking prevents duplicate drawers when agents mine the same file concurrently (#784, #826)
Hybrid closet+drawer retrieval — closets boost ranking, never gate results (#795)
Stop hooks from making agents write in chat — saves tokens on every turn (#786)
Strip system tags, hook output, and Claude UI chrome from drawers before filing (#785)
Verbatim-safe strip_noise scoped to Claude Code JSONL only (#785)
Prevent diary entry ID collisions via microsecond timestamp and full content hash (#819)
Auto-rebuild stale drawers via NORMALIZE_VERSION schema gate
Enforce atomic topics in closets and extract richer pointers
Sync version.py to match pyproject.toml (#820)
Remove unused main import from mempalace/__init__.py (#827)
README audit — fix 7 stale claims (tool count, version badge, wake-up token cost, dialect.py lossless disclaimer, pyproject.toml version) with 42 regression-guard tests (#835)

Improvements

Optimize entity detection with regex caching and pre-compilation (#828)
Extract locked filing block into helper to keep mine_convos under C901 complexity

Documentation

Add docs/CLOSETS.md — closet layer overview
Fix stale milla-jovovich/* org URLs in website and plugin manifests (#787)
Fix remaining stale org URLs in contributor docs (#808)
Rewrite README.md and mempalaceofficial.com benchmark pages to remove category-error cross-system comparisons (R@5 retrieval recall had been listed next to competitor QA accuracy under one column), remove the retracted "+34% palace boost" claim from the surfaces where it had remained, replace the 100% Haiku-rerank headline with the honest held-out 98.4% R@5, drop the LoCoMo 100% top-50 row (retrieval-bypass artefact), and fix the broken aya-thekeeper/mempal reproduction URL (#875)
Add docs/HISTORY.md as the canonical home for corrections, retractions, and public notices; move the 2026-04-07 "Note from Milla & Ben" and the 2026-04-11 impostor-domain notice out of README.md
Add v3.3.0 reproduction result JSONLs and the deterministic seed=42 50/450 LongMemEval split under benchmarks/ — every BENCHMARKS.md claim reproduces exactly

Internal

Add test coverage for mine_lock, closets, entity metadata, BM25, and diary
Verify mine_lock via disjoint critical-section intervals
Serialize mine_lock concurrency test with multiprocessing
Make diary state path assertion platform-neutral
Add TestTunnels coverage for cross-wing tunnel operations
Ruff format with CI-pinned version (0.4.x); format mempalace/palace.py

3.2.0 — 2026-04-12

Packaging

Remove chromadb<0.7 upper bound — unblocks installs against chromadb 1.x palaces (#690)
Bump version to 3.2.0 across pyproject.toml, mempalace/version.py, README badge, and OpenClaw SKILL (#761)

Security

Harden palace deletion, WAL redaction, and MCP search input handling (#739)
Consistent input validation, argument whitelisting, concurrency safety, and WAL fixes (#647)
Remove hardcoded credential paths from benchmark runners (#177)
Remove global SSL verification bypass in convomem_bench (#176)

Bug Fixes

Parse Claude.ai privacy export with messages key and sender field (#685, #677)
Detect mtime changes in _get_client to prevent stale HNSW index (#757)
Hash full content in tool_add_drawer drawer ID — stable re-mines (#716)
Remove 10k drawer cap from status display (#707, #603)
Correct typo in entity_detector interactive classification prompt (#755)
Prevent convo_miner from re-processing 0-chunk files on every run (#732, #654)
Remove silent 8-line AI response truncation in convo_miner (#708, #692)
Store full AI response in convo_miner exchange chunking (#695)
Fix mine --dry-run TypeError on files with room=None (#687, #586)
Skip arg whitelist for handlers accepting **kwargs (#684, #572)
Allow Unicode in sanitize_name() — Latvian, CJK, Cyrillic (#683, #637)
Auto-repair BLOB seq_ids from chromadb 0.6→1.5 migration (#664)
Remove no-op ORT_DISABLE_COREML env var (#653, #397)
Disambiguate hook block reasons to name MemPalace explicitly (#666)
Use epsilon comparison for mtime to prevent unnecessary re-mining (#610)
Correct token count estimate in compress summary (#609)
Implement MCP ping health checks (#600)
Align cmd_compress dict keys with compression_stats() return values (#569)
Skip unreachable reparse points in detect_rooms_from_folders on Windows (#558)
Prevent HNSW index bloat from duplicate add() calls (#544, #525)
Purge stale drawers before re-mine to avoid hnswlib segfault (#544)
Mitigate system prompt contamination in search queries (#385, #333)
Count Codex user_message turns in _count_human_messages (#373, #347)
Paginate large collection reads and surface errors in MCP tools (#371, #339, #338)
Expand ~ in split command directory argument (#361)
Ignore wait_for_previous argument to support Gemini MCP clients (#322)
Close KnowledgeGraph SQLite connections in test fixtures (#450)
Remove duplicate cache variable declarations in mcp_server.py (#449)
Add --yes flag to init instructions for non-interactive use (#682, #534)
Add mcp command with setup guidance (#315)

New Features

i18n support — 8 languages (en, es, fr, de, ja, ko, zh-CN, zh-TW) (#718)
New MCP tools: get/list/update drawer, hook settings, export (#667, #635)
mempalace migrate — recover palaces from different ChromaDB versions (#502)
Add OpenClaw/ClawHub skill (#491)
Backend seam for pluggable storage backends (#413)

Improvements

Disable broken auto-bump workflow (#414)
Improve agent readiness — AGENTS.md, dependabot, CODEOWNERS, labels (#497)

Documentation

Add CLAUDE.md and mission/principles to AGENTS.md (#720)
Add VitePress documentation site (#439)
Add warning about fake MemPalace websites (#598)
Fix stale org URLs and PR branch target in contributor docs (#679)
Fix misaligned architecture diagram (#734, #733)
Add ROADMAP.md — v3.1.1 stability patch and v4.0.0-alpha plan

Internal

ruff format convo_miner.py (#741)
ruff format all Python files (#675)
CI: trigger tests on develop branch PRs and pushes (#674)
CI: fix GitHub Pages publishing (#691)

3.1.0 — 2026-04-09

Security

Harden inputs, fix shell injection, optimize DB access (#387)
Sanitize SESSION_ID in save hook to prevent path traversal (#141)
Sanitize error responses and remove sys.exit from library code (#139)
Shell injection fix in hooks, Claude Code mining, chromadb pin (#114)

Bug Fixes

MCP null args hang, repair infinite recursion, OOM on large files (#399)
Release ChromaDB handles before rmtree on Windows (#392)
Use os.utime in mtime test for Windows compatibility (#392)
Negotiate MCP protocol version instead of hardcoding (#324)
Use upsert and deterministic IDs to prevent data stagnation (#140)
Make drawer_id deterministic for idempotent writes (#387)
Honest AAAK stats — word-based token estimator, lossy labels (#147)
Room detection checks keywords against folder paths (#145)
Use actual detected room in mine summary stats (#165)
Honour --palace flag in mcp_server (#264)
Preserve default KG path when --palace not passed (#270)
--yes flag skips all interactive prompts in init (#123)
Repair command, split args, Claude export, room keywords (#119)
Replace Unicode separator in convo_miner.py for Windows compatibility (#129)
Coerce MCP integer arguments to native Python int (#84)
Batch ChromaDB reads to avoid SQLite variable limit (#66)
Respect nested .gitignore rules during mining (#78)
Narrow bare except Exception to specific types where safe (#54)
Mark MD5 as non-security in miner drawer ID generation (#53)
Remove dead code and duplicate set items in entity_registry.py (#42)
Silence ChromaDB telemetry warnings and CoreML segfault on Apple Silicon (#236)
Unify package and MCP version reporting (#16)
Fix broken AAAK Dialect link in README (#238)
Update input prompt for entity confirmation (#83)
Preserve CLI exit codes, log tracebacks, sanitize search errors (#139)
Enable SQLite WAL mode and add consistent LIMIT to KG timeline (#136)
Add limit=10000 safety cap to all unbounded ChromaDB .get() calls (#137)
Re-mine modified files, idempotent add_drawer, cleanup ChromaDB handles (#140)
Resolve formatting, regression logic, and pytest defaults (#270)
Use parse_known_args to allow importing mcp_server during pytest (#270)

New Features

Package MemPalace as standard Claude and Codex plugins (#270)
Add OpenAI Codex CLI JSONL normalizer (#61)
Add Codex plugin support with hooks, commands, and documentation (#270)
Add command documentation for help, init, mine, search, and status (#270)

Improvements

Cache ChromaDB PersistentClient instead of re-instantiating per call (#135)
Tighten chromadb version range and add py.typed marker (#142)
Consolidate split known-names config loading (#22)
CI: add separate jobs for Windows and macOS testing
CI: Upgrade GitHub Actions for Node 24 compatibility (#55)

Documentation

Add Gemini CLI setup guide and integration section (#106)
Add beginner-friendly hooks tutorial (#103)
Align MCP setup examples with shipped server (#21)
Honest README update — own the mistakes, fix the claims

Internal

Expand test coverage from 20 to 92 tests, migrate to uv (#131)
Add scale benchmark suite — 106 tests (#223)
Increase test coverage from 30% to 85%, fix Windows encoding bugs (#281)
Add WAL mode and entity timeline limit assertions
Add coverage for file_already_mined mtime check

3.0.0 — 2026-04-06

Initial public release.

Palace architecture with day-based rooms, drawers (verbatim), and closets (searchable index)
AAAK compression dialect for memory folding
Knowledge graph with entity detection and timeline queries
MCP server for Claude, Codex, and Gemini integration
CLI: init, mine, search, status, compress, repair, split
Benchmark suite with recall and scale tests
README with MCP flow, local model flow, and specialist agent documentation

Uh oh!

FilesExpand file tree

CHANGELOG.md

Latest commit

History

CHANGELOG.md

File metadata and controls

Changelog

Unreleased

Bug Fixes

3.8.0 — 2026-08-20

Features

Bug Fixes

3.7.1 — 2026-08-12

Bug Fixes

3.7.0 — 2026-08-11

Features

Performance

Bug Fixes

Documentation

Internal

3.6.0 — 2026-07-14

Features

Performance

Bug Fixes

Documentation

Internal

3.5.0 — 2026-06-22

Features

Performance

Bug Fixes

Internal

3.4.1 — 2026-06-14

Features

Bug Fixes

3.3.6 — 2026-05-24

Features

Bug Fixes

Performance

Internal

3.3.5 — 2026-05-09

Bug Fixes

Documentation

Internal

3.3.4 — 2026-04-30

Added

Bug Fixes

3.3.3 — 2026-04-23

Bug Fixes

Improvements

Added

Known — deferred to v3.3.4

3.3.2 — 2026-04-19

Bug Fixes

Improvements

Documentation

Internal

3.3.1 — 2026-04-16

New Features

Bug Fixes

Improvements

Documentation

Internal

3.3.0 — 2026-04-13

New Features

Bug Fixes

Improvements

Documentation

Internal

3.2.0 — 2026-04-12

Packaging

Security

Bug Fixes

New Features

Improvements

Documentation

Internal

3.1.0 — 2026-04-09

Security

Bug Fixes