You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+17Lines changed: 17 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,8 +8,24 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
8
8
9
9
## [Unreleased]
10
10
11
+
---
12
+
13
+
## [3.8.0] — 2026-08-20
14
+
15
+
Large palaces get fast and stay small: both storage backends lost their palace-wide read paths, a proxied agent session no longer loads a storage stack it never uses, and agents gained a background watcher so coordination stops stalling on nobody listening.
16
+
17
+
### Features
18
+
19
+
- **`mempalace logstream watch` — a background watcher an agent can be woken by.** `logstream wait` is a primitive, not a watcher: it caps at five minutes and reports a timeout, so every caller ended up writing the same re-arm loop and each one had to remember to carry its cursor forward. Most did not, and coordinated tasks stalled on nobody listening rather than on the work. `watch` owns the loop and the cursor, and exits on a match so any harness that can background a process and react to its exit gets woken — `0` when it printed a match, `2` on `--idle-exit-ms`, `130` when interrupted; only `0` means "you have mail". Two filters a watcher needs are things `list_events` cannot express, because its SQL is single-valued and positive-only: repeating a flag means "or" (`--type task.request --type patch.ready` wakes for either and stays quiet for everything else), and `--agent <id>` expands to `--to-agent <id> --exclude-from-agent <id>`. That exclusion is not cosmetic — `to_agent=<you>` deliberately also matches `*` broadcasts, and your own broadcasts are broadcasts, so a watcher without it wakes itself every time it posts a status. `--state-file` persists the cursor so a restart resumes exactly, advancing past events that were examined and rejected rather than only matches; a cursorless first run starts at the tip like the SSE live-tail rather than replaying weeks of fleet history it cannot tell is stale, and says so on stderr. `--follow` keeps the process alive past the first match and emits NDJSON, since repeated indented documents on one stream are not parseable JSON. The invariant throughout is that a restart may cost a duplicate, never a missed delegation. (#2315)
20
+
- **The monitoring protocol is documented, including the cursor rule that costs real events.** Events are ordered by append order (`ORDER BY rowid`), not wall clock, so a peer's event is appended whenever it syncs and can already be older than a timestamp high-water mark — resuming with `since_created_at` therefore drops late-arriving cross-replica events permanently. Measured on a live fleet: a windows-origin event created `09:10:48Z` was ingested *after* a mac-origin event created `09:13:21Z`, in a single fifty-event window. `list_events`' docstring already said `since_event_id` was "the precise cursor … regardless of timestamp ties"; it just never reached an agent. That rule now appears in the `mempalace_event_list` and `mempalace_event_wait` tool descriptions at the point of use, both `since_created_at` parameters are marked "NOT a resume cursor", and `coordination-protocol.md` gained a monitoring section covering the modes, the announce-your-watch convention, and declaring when you are *not* watching — a false watcher is worse than a declared-absent one, because the requester stops looking for a human to nudge. The system-prompt snippet every agent copies was updated with it. (#2315)
21
+
11
22
### Bug Fixes
12
23
24
+
- **`list_drawers` and the tunnel tools stop cold-loading the vector index on a chroma palace.** Both paged `col.get()`, which opens the collection and loads HNSW before answering a question that is pure metadata: on a 1.7 GB / 165k-drawer palace `list_drawers` with no filter and `limit=20` took **36.6 s**, and `find_tunnels` died at 29.7 s with an internal tool error. `chroma.sqlite3` already holds everything those calls need, so they now read it directly and never open the collection. Two traps had to be avoided to make that a win rather than a trade. `chroma:document` lives in `embedding_metadata` alongside the loci, so an unqualified join drags the palace's entire verbatim text — 162 MB across 331k rows here — into memory to render one page; documents are excluded from the scan and the displayed page is hydrated separately. And `embedding_id` is only indexed as the second column of `UNIQUE (segment_id, embedding_id)`, so a join filtering on it alone degenerates into a full scan of `embedding_metadata`: 5.7 s for a single twenty-row page, against 0.003 s once the segment is sought first and `embedding_metadata`'s `(id, key)` primary key second. The wing/room filter is pushed into SQL rather than applied afterwards. Measured on the same palace, `list_drawers(wing=…, limit=20)`: **2.31 s and 1148 MB peak RSS → 0.01 s and 86 MB**. `find_tunnels`' `recent` field survives the move via a `MAX(date)` on the grouped read, and a missing or unreadable palace still reports a diagnostic rather than looking like a palace with no tunnels. (#2314)
25
+
-**`sqlite_exact` search and status stop scanning the whole palace.**`query()` was exact cosine over every row — selecting `id`, `document`, `metadata_json` and `embedding`, JSON-parsing metadata, then dotting in a Python loop — which `mempalace_search` does twice, once for drawers and once for closets. `status` paged every metadata row because the backend had no `facet_counts` and the taxonomy read was chroma-only. Ranking now runs vectorized over the embedding column with only the top-k documents hydrated, the embedding matrix and the `wing` / `room` / `source_file` columns are cached on the long-lived backend handle and invalidated on write, and `facet_counts` plus a grouped `json_extract` read serve status, `list_wings` and `list_rooms`. Exact cosine ranking is unchanged. Measured on a live 167k-drawer / 166k-closet palace: `mempalace_status`**6997 ms → 1045 ms**, warm `mempalace_search`**6364 ms → 210–1600 ms**, wing-scoped search 3910 ms → 1453 ms. (#2308)
26
+
-**The remaining palace-wide reads on `sqlite_exact` are gone.**`get(ids=…)` scanned every row and then did a dict lookup, `include=["metadatas"]` still selected documents and embeddings, and equality filters with `LIMIT`/`OFFSET` were applied in Python — so `list_drawers` at `limit=20` took 2752 ms because it loaded every document in the palace to show twenty previews. Those now go to SQL, pagination walks metadata only to collapse logical drawers and compute the total, and the page's previews are hydrated by id. `status`, `list_wings`, `list_rooms` and `get_taxonomy` share one grouped result behind a 5 s cache dropped on writes, and `graph_stats` gained the same sqlite reconstruction the chroma path uses. (#2311)
27
+
-**`wing`, `room` and `hall` are indexed places rather than JSON scanned palace-wide.**`sqlite_exact` stored them only inside `metadata_json`, so every structural question — taxonomy, `graph_stats`, `list_drawers where=wing`, facet counts — meant `json_extract` across the corpus. They are now VIRTUAL generated columns over the same JSON with a composite `(collection_id, wing, room, hall)` index, which existing palaces pick up on their next writable open. VIRTUAL is the point: the 1.6 GB embedding blob table is not rewritten, so the migration walks metadata once rather than copying a table. Read-only opens against a pre-migration palace fall back to `json_extract` unchanged. (#2313)
28
+
- **`embeddinggemma` no longer produces silently unusable vectors on Apple Silicon.** With the default `embedding_device=auto`, `_AUTO_ORDER` puts CoreML ahead of CPU, and CoreML returns NaN or all-zero vectors for this model without raising — ONNX Runtime only warns on stderr about how few nodes it could take. Same model and input: CoreML gave 9216 NaN in the hidden state and a 768-zero pooled vector with norm 0.0, against a healthy norm on CPU, reproduced under two onnxruntime versions and two graph partitionings. Nobody opted into this, and a `repair rebuild-index` under those conditions rewrites the whole palace with vectors that are indistinguishable from healthy ones once stored. Two layers now: a per-model provider denylist keeps `auto` from selecting CoreML for embeddinggemma at all, and the vectors are checked for a **finite and non-zero** norm before use — an all-zero vector passes any `isnan()` test and is exactly as unusable, so a NaN-only check would have looked like a fix while missing the observed case. Anyone auditing an existing palace should look for zero-norm vectors, not NaN. CUDA and DirectML are untouched, and other models were never affected. (#2283)
13
29
- **A proxied MCP session no longer loads the storage stack it never uses.** `mempalace-mcp` is spawned once per agent session, and whenever a hub is running every one of those processes is a pure proxy — `_dispatch_stdio_request` forwards each JSON-RPC request over HTTP and local storage is never touched. Importing `mempalace.mcp_server` to do that cost ~77 MB regardless, because chromadb (~61 MB by itself), numpy, pydantic, grpc and opentelemetry are all imported at module scope: a fleet of 50 agents spent ~3.9 GB holding proxies that did no work. The `mempalace-mcp` entry point is now `mempalace.mcp_proxy`, which imports only the standard library plus `config` and `server_registry`; a proxied session runs at ~17 MB measured end to end (~24 MB import weight against ~80 MB), and the full server is imported lazily the first time this process must answer a request itself. Anything that is not a plain stdio session — another transport, a server-side flag, an unrecognised argument — goes straight to the full server unchanged. The local fallback is preserved exactly, including the rule that a mutating call which failed mid-flight is never replayed locally, and it now says so: the first `tools/call` served locally carries a notice in `result.content` telling the driving agent its memory backend lost the hub and this process is holding the index. Restoring stdout on that lazy import is required, not cosmetic — importing the server performs `os.dup2(2, 1)`, so responses would otherwise be written to stderr and the session would hang. (#2312)
14
30
- **A long-running MCP server no longer grows by ~440 MB per collection open.** `ChromaBackend._client` keys its client cache on `chroma.sqlite3`'s inode and mtime to detect writes by another process, but constructing a `chromadb.PersistentClient` writes to that file itself, and so do the collection opens that follow — so the stamp taken at construction time was already stale by the time the next open compared against it. Every search opens `mempalace_drawers` and then `mempalace_closets`, and the first open moved the mtime the second one checked, so the cache missed on essentially every request: each search rebuilt the client and reloaded both HNSW segments, and the displaced client was dropped from the dict without being closed, so its native index memory was never returned. On a palace of ~165k vectors that was ~650 MB of resident growth per search; a server left running for an afternoon reached 3.9 GB. The freshness stat is now re-baselined once the backend's own operation finishes — including writes through `ChromaCollection`, so the file-a-drawer-then-search cycle stops reloading the index too — which makes the recorded value mean "`chroma.sqlite3` as this backend last left it", and a genuine external change still rebuilds. The rebuild path also closes the client it replaces. Measured over eight searches on the same palace: 951 MB → 3522 MB before, 940 MB → 965 MB after. An external write landing *while* one of our own operations is in flight is absorbed into the new stamp and picked up on the next change; mtime cannot distinguish writers, and `PRAGMA data_version` does not help because chromadb's own connection reads as foreign to a probe connection. (#2307)
15
31
- **`mempalace_mesh_peers` answers with the real estate from every transport, not just the hub.** The peer sync loop is started only by `_serve_http`, and the estate it builds (`_PEER_SYNC_STATE`, `_KNOWN_PROFILES`) lives in process memory — but the tool that reports it ships in every transport, including the stdio servers agents actually connect through. Those processes never run a sync round, so they answered from a permanently empty estate: each peer reduced to a bare `name` and `url` with no `reachable`, `last_success_at`, `remote_version_vector` or `profile`, `origin_profiles` holding only this node, and configured peers misreported as `unnamed_origins` because their `replica_id` was never learned — all while the hub next door had the complete picture. The sync loop now publishes the estate to `mesh_state.json` in the per-palace server state directory (0600, write-and-rename) after every round, and `_mesh_peers_payload` merges it underneath any in-process state, so the syncing process still trusts its own fresher reading. A new `estate_source` field says whether the status was observed in this process or published by the hub, when it was published, and whether that hub is still alive — a crashed hub leaves a last-known-good estate that is shown but never read as live. peers.json tokens never reach the file. (#2309)
Copy file name to clipboardExpand all lines: integrations/openclaw/SKILL.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
2
name: mempalace
3
3
description: "MemPalace — Local AI memory with 96.6% recall. Semantic search, temporal knowledge graph, palace architecture (wings/rooms/drawers). Free, no cloud, no API keys."
0 commit comments