fix: PreCompact hook reads a persisted live session journal (#334, v1.108.57)

The hook-precompact CLI runs in a separate process from the MCP server, so it
read a fresh empty in-process SessionJournal and emitted a healthy-looking but
empty "Files explored: 0 | Searches: 0" snapshot at compaction time.

The live server now persists a compact journal snapshot to an atomically
written _session_live.json in CODE_INDEX_PATH (save_live_journal, throttled,
best-effort, independent of session_resume). The hook reads it back
(snapshot_from_live) and renders it through a shared _render_snapshot() so the
out-of-process snapshot matches the live tool byte-for-byte, including landmark
enrichment seeded from the persisted journal. When no live journal is readable,
the hook emits an explicit "no live session journal" message instead of a
zero-state snapshot. New file write disclosed in the README; disable with
JCODEMUNCH_LIVE_JOURNAL=0.

Reported by @mmashwani with a source-cited probe of the process boundary. 10
tests in tests/test_v1_108_57.py incl. a real cross-process check; full suite
4574 passed / 10 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: jgravelle

CHANGELOG.md

@@ -4,6 +4,32 @@ All notable changes to jcodemunch-mcp are documented here.
 
 ## [Unreleased]
 
+## [1.108.57] - 2026-06-17 - PreCompact hook reads a persisted live session journal (#334)
+
+### Fixed
+
+- **#334 — the PreCompact CLI hook can now see real session state.** The
+  `hook-precompact` command runs as a separate process from the MCP server, so
+  it imported `get_session_snapshot()`, read a fresh process-local
+  `SessionJournal`, and emitted a healthy-looking but empty
+  `Files explored: 0 | Searches: 0` snapshot — failing the one thing it exists to
+  do. The live server process now persists a compact journal snapshot to a
+  small, atomically written `_session_live.json` in `CODE_INDEX_PATH`
+  (`save_live_journal`, throttled to at most once every ~2s on the tool-call
+  path, best-effort, independent of `session_resume`). The hook reads it back
+  (`snapshot_from_live`) and renders it through the same formatter the live tool
+  uses — so the out-of-process snapshot now matches what the running server
+  sees, including structural-landmark enrichment seeded from the persisted
+  journal. When no live journal is readable, the hook emits an explicit
+  "no live session journal" message instead of an apparently-successful
+  zero-state snapshot. The snapshot renderer is factored into a shared
+  `_render_snapshot()` so the live tool and the hook stay byte-identical.
+  Disable the live journal with `JCODEMUNCH_LIVE_JOURNAL=0` (disclosed in the
+  README's background-behavior section). Reported by @mmashwani with a
+  source-cited probe of the process boundary. 10 tests in
+  `tests/test_v1_108_57.py`, including a real cross-process check (seed a
+  journal in one process, read it from a fresh `hook-precompact` subprocess).
+
 ## [1.108.56] - 2026-06-17 - no-change freshness refresh, cache-hit hardening, subset-scoped pruning (#330, #331, #333)
 
 ### Fixed

CLAUDE.md

@@ -1,6 +1,7 @@
 # jcodemunch-mcp — Project Brief
 
 ## Current State
+- **Version:** 1.108.57 — #334 (reported by @mmashwani). The `hook-precompact` CLI path runs in a SEPARATE process from the MCP server, so it imported `get_session_snapshot()`, read a fresh process-local `SessionJournal`, and emitted a healthy-looking but empty `Files explored: 0 | Searches: 0` snapshot. Fix (his durable option 1 + explicit-fallback option 3): the live server persists a compact journal snapshot to `<CODE_INDEX_PATH>/_session_live.json` via new `save_live_journal()` in `tools/session_state.py` (throttled ≤1/~2s on the tool-call recording path in `server.py` via `_maybe_flush_live_journal`, best-effort, per-PID temp + atomic replace, independent of `session_resume`, off via `JCODEMUNCH_LIVE_JOURNAL=0`). The hook reads it back via new `snapshot_from_live()` in `tools/get_session_snapshot.py` and renders through a factored shared `_render_snapshot()` (so live tool + hook are byte-identical); landmark enrichment now accepts an optional `context=` so it works out-of-process too. No live journal → hook emits an explicit "No live jCodeMunch session journal" message, not a zero-state snapshot. New file write is DISCLOSED in README background-behavior section (PyPI-compliance surface). 10 tests `tests/test_v1_108_57.py` incl. a real cross-process subprocess check. **#332 (named deep-audit budget contexts) still open — 2.x-framed enhancement, deferred.**
 - **Version:** 1.108.56 — issue batch #330/#331/#333 (all reported by @mmashwani, source-cited). **#330**: `FreshnessProbe` flags results `stale_index` on stored-SHA-vs-live-HEAD mismatch, but all three `index_folder` no-change return paths (`tools/index_folder.py`: watcher fast-path pre-read, mtime-only, plain-incremental) returned "No changes detected" WITHOUT refreshing the stored `git_head` — so a commit touching only non-indexed files left otherwise-current symbols stale-flagged right after a successful incremental run. New `_refresh_git_head_if_advanced()` writes a metadata-only delta (empty changed/new/deleted) advancing `git_head`+`indexed_at` when live HEAD moved; wired into all three paths (mtime-only folds `git_head` into its existing `incremental_save`). Best-effort (swallows failures). **#331**: `_result_cache_get` tolerates a cached result without `_meta`, but the cache-hit return path assumed `_meta` existed when stamping `timing_ms`/`cache_hit` → `KeyError("_meta")`; AND the `server.py` dispatcher rendered ANY `KeyError` (incl. ones from inside tool code) as "Missing required argument", masking internal bugs as caller-schema errors. Two layers: cache hits `setdefault` the `_meta` envelope; the dispatcher `KeyError` handler walks `e.__traceback__` to the originating frame and only reports missing-argument when it's server.py's own arg-extraction frame, else logs + returns "Internal error processing <tool>" with the key. (Schema input-validation already preempts genuine missing required args, so the missing-arg branch is now a rare fallback.) **#333**: `index_folder(paths=[...])` diffed the subset against the ENTIRE stored index (`detect_changes_with_mtimes` computes `deleted = old_set - new_set`), so every unlisted indexed file was pruned. `resolve_explicit_paths` now also returns `requested_rels` (root-relative listed entries, incl. dirs + on-disk-deleted paths); the incremental path rescopes `deleted` to only files covered by that subset (`fp == req or fp.startswith(req+"/")`); listing root (`.`/`""`) keeps full-corpus semantics; the `No source files found` guard lets a deletion-only subset through so a listed-but-deleted file is still pruned. Rescope sits on the shared `deleted` list → branch-delta save covered too. Mirrors jdocmunch #31. 9 tests `tests/test_v1_108_56.py`.
 - **Version:** 1.108.55 — issue batch #327/#328/#329 (all reported by @mmashwani, source-cited). **#327**: turn-budget idle-gap reset only fired lazily in `record_output` (post-dispatch), so in-execution readers (`plan_turn` budget advisor via `percent_used()`, `should_compact()`) reported the PREVIOUS turn's exhausted counter after an idle gap (143.4% observed); reset factored into `_maybe_reset_locked()` + called from both readers; reader resets don't advance `_last_call_ts`; class docstring now documents the process-global-across-repos scope + rolling-budget semantics under continuous use. **#328**: `token_budget` packing (all 3 paths: BM25/semantic/fusion) charged source-body `byte_length` even for compact/standard rows → 84 rows for max_results=18 and `tokens_used` describing unreturned code; new `_packing_cost_bytes()` charges encoded-row size for compact/standard (full keeps §1.2 materialized byte_length — existing tests pin that); `tokens_used`/`tokens_remaining` from payload; new `_row_summary()` empties summaries that merely echo `signature[:120]` (signature_fallback persisted at index time when no AI summarizer) — "Class Foo"-style fallbacks preserved; "budget overrides max_results" contract unchanged, schema+docstring now say compact rows are cheap so budgets admit many rows. **#329**: search_text's 200-char regex / 500-char plain caps documented in schema+docstring; checks factored into `validate_query_args()` and hoisted into `call_tool` BEFORE strict-freshness + auto-watch so doomed calls reject in ms (was 29.3s behind an auto-watch reindex). 15 tests `tests/test_v1_108_55.py`.
 - **Version:** 1.108.54 — `list-repos` + `delete-index` honor `CODE_INDEX_PATH`. Both CLI dispatches called into the store with no storage path (→ always `~/.code-index`), unlike every other index verb. Found via CI run 27341297693: **CI had been red since v1.108.50** — `test_delete_index_cli_roundtrip` relied on `CODE_INDEX_PATH` for isolation, so on CI (empty home store) it saw `[]` and failed, while locally it "passed" by reading the REAL store and **deleting `repos[0]` of `~/.code-index` on every full-suite run** (.50/.51/.52/.53 each sacrificed one real index; all reindexable, munch indexes survived). Fix = thread `os.environ.get("CODE_INDEX_PATH")` at both dispatch sites; test hardened with an explicit isolation guard (exactly-one-repo + source_root match) so a regression fails loudly instead of destroying an index. Lessons re-burned: env-coupled tests + let-CI-gate-before-tagging ([[feedback_env_coupled_tests_need_ci]]). Known follow-up: other tests leak `test_*` junk indexes into the real store (additive-only, hygiene not data-loss).
@@ -253,6 +254,7 @@ Tree-sitter grammar lacks clean named fields for these — custom regex extracto
 | `GEMINI_EMBED_TASK_AWARE` | 1 | Set `0`/`false`/`no`/`off` to disable task-type hints (`RETRIEVAL_DOCUMENT` / `CODE_RETRIEVAL_QUERY`) when using Gemini embeddings |
 | `JCODEMUNCH_CROSS_REPO_DEFAULT` | 0 | Set 1 to enable cross-repo traversal by default in find_importers, get_blast_radius, get_dependency_graph |
 | `JCODEMUNCH_EVENT_LOG` | — | Set `1` to write `_pulse.json` on every tool call (per-call activity signal for dashboards) |
+| `JCODEMUNCH_LIVE_JOURNAL` | 1 | (v1.108.57) Set `0`/`false`/`no`/`off` to disable the live session-journal write (`<CODE_INDEX_PATH>/_session_live.json`). On by default so the out-of-process PreCompact hook can read real session state (#334); throttled ≤1/~2s, paths+queries only, no file contents. |
 | `JCODEMUNCH_PARSE_CACHE` | — | Shared directory for the content-addressed parse cache (v1.108.40). Point all seats on a multi-home-dir box at the same path so identical files parse once across seats. Unset = disabled (no caching). |
 | `JCODEMUNCH_PARSE_CACHE_MAX_ROWS` | 50000 | (v1.108.41) Row cap for the shared parse cache; FIFO-trimmed oldest-first by rowid after each write (stale-content/stale-version rows go first). `<= 0` disables the cap (unbounded). |
 | `JCODEMUNCH_ORG_ID` | — | Org identifier for the team-SKU rollup (`org-report` / `org-rollup`) |

README.md

@@ -110,9 +110,9 @@ is a byte the agent doesn't pay to read.
 <!-- WHATSNEW:START -->
 #### What's new
 
+- **[v1.108.57](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.57)** (2026-06-17) — PreCompact hook reads a persisted live session journal (#334)
 - **[v1.108.56](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.56)** (2026-06-17) — no-change freshness refresh, cache-hit hardening, subset-scoped pruning (#330, #331, #333)
 - **[v1.108.55](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.55)** (2026-06-12) — turn-budget reset, honest budget packing, fast search_text rejection (#327, #328, #329)
-- **[v1.108.54](https://github.com/jgravelle/jcodemunch-mcp/releases/tag/v1.108.54)** (2026-06-11) — list-repos / delete-index honor CODE_INDEX_PATH
 <!-- WHATSNEW:END -->
 
 ![License](https://img.shields.io/badge/license-dual--use-blue)
@@ -319,6 +319,7 @@ Everything jCodeMunch does beyond answering a tool call is listed here. All of i
 - **Anonymous savings telemetry.** The server periodically sends a random anonymous ID plus aggregate token-savings counters to the project's public community meter. No code, no file paths, no repo names, no PII — counters only. Opt out with `share_savings: false` in `config.jsonc` or `JCODEMUNCH_SHARE_SAVINGS=0`.
 - **Agent hooks.** `init` / `install` can write hook entries (auto-reindex on edit, read-interception nudges) into your MCP client's settings. They're offered during the interactive flow, shown before writing, and fully removed by `uninstall`.
 - **Local index storage.** Indexes live at `~/.code-index/` (override with `CODE_INDEX_PATH`). Delete the directory and every trace of indexing is gone.
+- **Live session journal.** While the server runs, it periodically writes a small `_session_live.json` in `~/.code-index/` recording the files and searches the agent touched this session (paths and query strings only, no file contents). It exists so the out-of-process PreCompact hook can restore session orientation after context compaction. Throttled, atomically written, overwritten in place; disable with `JCODEMUNCH_LIVE_JOURNAL=0`.
 
 The base package makes no other network calls and leaves no other persistent processes. AI-summary extras call their configured provider's API only when you enable them — see the extras matrix under [Start fast](#start-fast).
 

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "jcodemunch-mcp"
-version = "1.108.56"
+version = "1.108.57"
 description = "Token-efficient MCP server for source code exploration via tree-sitter AST parsing"
 readme = "README.md"
 requires-python = ">=3.10"

src/jcodemunch_mcp/cli/hooks.py

@@ -424,24 +424,48 @@ def run_precompact() -> int:
     except (json.JSONDecodeError, ValueError):
         return 0
 
-    # Build snapshot in-process (no MCP round-trip needed)
+    # The hook runs as a SEPARATE process from the MCP server, so the in-process
+    # SessionJournal is empty (#334). Read the live journal the server persists
+    # incrementally first; fall back to the in-process journal (covers embedded
+    # invocations); finally emit an explicit "no live session" message rather
+    # than a misleading zero-state snapshot.
+    snapshot_text = ""
+    live_context = None
     try:
-        from jcodemunch_mcp.tools.get_session_snapshot import get_session_snapshot
-        snapshot_result = get_session_snapshot()
-        snapshot_text = snapshot_result.get("snapshot", "")
+        from jcodemunch_mcp.tools.get_session_snapshot import snapshot_from_live
+        live = snapshot_from_live()
+        if live:
+            snapshot_text = live.get("snapshot", "")
+            live_context = live.get("_context")
     except Exception:
-        return 0  # Snapshot failure must not block compaction
+        snapshot_text = ""
 
     if not snapshot_text:
-        return 0
+        try:
+            from jcodemunch_mcp.tools.get_session_snapshot import get_session_snapshot
+            snap = get_session_snapshot()
+            structured = snap.get("structured", {})
+            if structured.get("total_files_explored") or structured.get("total_searches"):
+                snapshot_text = snap.get("snapshot", "")
+        except Exception:
+            snapshot_text = ""
 
-    # Enrich with structural landmarks (PageRank top-N) and recently-changed symbols
-    try:
-        landmarks = _build_landmark_section()
-        if landmarks:
-            snapshot_text += landmarks
-    except Exception:
-        pass  # Landmark enrichment must not block compaction
+    used_fallback = False
+    if not snapshot_text:
+        # Honest fallback — never emit a healthy-looking zero-state snapshot.
+        snapshot_text = _precompact_no_journal_message()
+        used_fallback = True
+
+    # Enrich with structural landmarks (PageRank top-N) and recently-changed
+    # symbols. Seed from the live journal context when we have one so landmarks
+    # work out-of-process too; skip entirely on the no-journal fallback.
+    if not used_fallback:
+        try:
+            landmarks = _build_landmark_section(context=live_context)
+            if landmarks:
+                snapshot_text += landmarks
+        except Exception:
+            pass  # Landmark enrichment must not block compaction
 
     # Return snapshot as hook output for context injection.
     # PreCompact has no hookSpecificOutput variant in Claude Code's schema,
@@ -453,18 +477,39 @@ def run_precompact() -> int:
     return 0
 
 
+def _precompact_no_journal_message() -> str:
+    """Explicit fallback when no live jCodeMunch session journal is readable.
+
+    The PreCompact hook runs in a separate process from the MCP server (#334);
+    when the server has not persisted a live journal this session, say so plainly
+    instead of emitting a `Files explored: 0` snapshot that looks successful.
+    """
+    return (
+        "## Session Snapshot (jCodemunch)\n"
+        "No live jCodeMunch session journal was available to this PreCompact hook "
+        "process. The hook runs separately from the MCP server, so it could not "
+        "read in-flight session state. If jCodeMunch tools were used this session, "
+        "their snapshot is unavailable here; otherwise no code exploration was "
+        "recorded. (Set JCODEMUNCH_LIVE_JOURNAL=0 only if you intend to disable "
+        "the live journal.)"
+    )
+
+
 # ---------------------------------------------------------------------------
 # Landmark enrichment helpers (Gap 4A — Structural Landmarks)
 # ---------------------------------------------------------------------------
 
-def _build_landmark_section(top_n: int = 20) -> str:
+def _build_landmark_section(top_n: int = 20, context: "dict | None" = None) -> str:
     """Build a compact landmarks + recently-changed section for PreCompact.
 
     Queries all indexed repos visible in the session journal's edited files,
     computes PageRank to find the most structurally central symbols, and
     cross-references the journal's edit log to surface recently-changed symbols.
 
-    Returns a markdown string to append to the snapshot, or "" if no data.
+    When ``context`` is supplied (e.g. the live journal read by the
+    out-of-process hook, #334) it is used instead of the empty in-process
+    journal. Returns a markdown string to append to the snapshot, or "" if no
+    data.
     """
     import logging
     logger = logging.getLogger(__name__)
@@ -477,8 +522,9 @@ def _build_landmark_section(top_n: int = 20) -> str:
         logger.debug("landmark imports failed", exc_info=True)
         return ""
 
-    journal = get_journal()
-    context = journal.get_context(max_files=50, max_queries=0, max_edits=50)
+    if context is None:
+        journal = get_journal()
+        context = journal.get_context(max_files=50, max_queries=0, max_edits=50)
     edited_files = [e["file"] for e in context.get("files_edited", [])]
     accessed_files = [f["file"] for f in context.get("files_accessed", [])]
 

src/jcodemunch_mcp/server.py

@@ -578,6 +578,43 @@ def _save_session_state() -> None:
 atexit.register(_save_session_state)
 
 
+# ---------------------------------------------------------------------------
+# Live journal persistence (#334) — feeds the out-of-process PreCompact hook.
+#
+# The hook (`jcodemunch-mcp hook-precompact`) runs in a separate process from
+# this server, so it sees a fresh, empty SessionJournal. We persist a compact
+# snapshot of the live journal to a small file the hook reads back. Writes are
+# throttled (not every tool call) and best-effort. Disable with
+# JCODEMUNCH_LIVE_JOURNAL=0.
+# ---------------------------------------------------------------------------
+
+_live_journal_lock = threading.Lock()
+_live_journal_last_flush = 0.0
+_LIVE_JOURNAL_MIN_INTERVAL_S = 2.0
+
+
+def _live_journal_enabled() -> bool:
+    val = os.environ.get("JCODEMUNCH_LIVE_JOURNAL", "").strip().lower()
+    return val not in {"0", "false", "no", "off"}
+
+
+def _maybe_flush_live_journal(journal) -> None:
+    """Throttled, best-effort flush of the live journal to disk (#334)."""
+    if not _live_journal_enabled():
+        return
+    global _live_journal_last_flush
+    now = time.monotonic()
+    with _live_journal_lock:
+        if now - _live_journal_last_flush < _LIVE_JOURNAL_MIN_INTERVAL_S:
+            return
+        _live_journal_last_flush = now
+    try:
+        from .tools.session_state import save_live_journal
+        save_live_journal(journal, base_path=os.environ.get("CODE_INDEX_PATH") or None)
+    except Exception:
+        logger.debug("live journal flush failed", exc_info=True)
+
+
 def _cleanup_mermaid_temp_startup() -> None:
     """Clean stale mermaid viewer temp files from previous sessions."""
     if not config_module.get("render_diagram_viewer_enabled", False):
@@ -4814,6 +4851,9 @@ async def call_tool(name: str, arguments: dict) -> list[TextContent]:
                                 "scanned_symbols": ne.get("scanned_symbols", 0),
                                 "timestamp": _t.time(),
                             })
+                # Persist a compact live snapshot so the out-of-process
+                # PreCompact hook can read real session state (#334). Throttled.
+                _maybe_flush_live_journal(journal)
             except Exception:
                 logger.debug("Journal recording failed", exc_info=True)