Skip to content

Commit 37ecc09

Browse files
jgravelleclaude
andcommitted
Tame watch-all CPU under WSL (#356)
watch-all pegged 15-30% CPU non-stop on WSL (reported by @Blainexi). Not a busy loop: watchfiles auto-enables polling whenever it detects WSL (inotify is unreliable across the boundary), and its default 300ms poll re-stats the whole watched tree several times a second, multiplied across every indexed repo. DefaultFilter drops node_modules/.git events but the polling walk still stats them, so the cost is structural. - Raise the poll-interval floor 300ms -> 1000ms and make it tunable: the awatch call now passes poll_delay_ms=_watch_poll_delay_ms(), resolved from JCODEMUNCH_WATCH_POLL_DELAY_MS (then WATCHFILES_POLL_DELAY_MS), else the 1000ms default; non-positive/garbage -> default. Ignored when native FS events are in use, so it is a no-op off WSL. - One-time WSL startup hint (_is_wsl()) naming the two levers: raise the poll delay, or set WATCHFILES_FORCE_POLLING=false for repos on the Linux filesystem to use native inotify (near-zero idle CPU). /mnt/* repos need polling. No INDEX_VERSION bump (runtime-only). New tests/test_v1_108_83.py (11); test_watch_all.py / test_watcher_lock.py / test_reindex_state.py green. Bump 1.108.82 -> 1.108.83. Files: watcher.py, watch_all.py, CHANGELOG, CLAUDE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent ccee60f commit 37ecc09

6 files changed

Lines changed: 188 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,39 @@
22

33
All notable changes to jcodemunch-mcp are documented here.
44

5+
## [1.108.83] - 2026-06-26 - Tame watch-all CPU under WSL (#356)
6+
7+
`watch-all` pegged 15-30% CPU continuously on WSL (reported by @Blainexi). Root
8+
cause is `watchfiles` behavior, not a busy loop: **watchfiles auto-enables
9+
polling whenever it detects WSL** (inotify is unreliable across the WSL
10+
boundary), and its default 300ms poll interval re-stats the entire watched tree
11+
several times a second — multiplied across every locally-indexed repo `watch-all`
12+
covers. `DefaultFilter` suppresses `node_modules`/`.git` *events* but the polling
13+
walk still stats them, so the cost is structural.
14+
15+
### Changed
16+
17+
- **Poll interval floor raised 300ms → 1000ms, and made tunable.** The `awatch`
18+
call now passes `poll_delay_ms=_watch_poll_delay_ms()`, resolved from
19+
`JCODEMUNCH_WATCH_POLL_DELAY_MS` (then watchfiles' own
20+
`WATCHFILES_POLL_DELAY_MS` as a fallback), else the 1000ms default. For a
21+
background *freshness* daemon a ~1s cadence is invisible; the CPU drop is
22+
roughly proportional to the increase. The param is ignored entirely when
23+
native FS events are in use (every non-WSL host), so this is a no-op off WSL.
24+
- **One-time WSL hint at `watch-all` startup** surfacing the two real levers:
25+
raise `JCODEMUNCH_WATCH_POLL_DELAY_MS`, or — for repos on the Linux filesystem
26+
— set `WATCHFILES_FORCE_POLLING=false` to use native inotify (near-zero idle
27+
CPU). Repos under `/mnt/*` need polling; moving them onto the Linux filesystem
28+
is faster all round.
29+
30+
New helpers `watcher._watch_poll_delay_ms()` / `watcher._is_wsl()`;
31+
`DEFAULT_WATCH_POLL_DELAY_MS = 1000`. No `INDEX_VERSION` bump (runtime-only).
32+
33+
### Tests
34+
35+
- `tests/test_v1_108_83.py` (11): poll-delay default/override/precedence/clamping
36+
+ WSL detection (non-linux, microsoft-tagged `/proc/version`, bare-metal).
37+
538
## [1.108.82] - 2026-06-24 - Exclude org_savings.db from list_repos
639

740
The team-SKU org-rollup store (`org_savings.db`, written by `org/store.py`) lives

CLAUDE.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# jcodemunch-mcp — Project Brief
22

33
## Current State
4+
- **Version:** 1.108.83 — Tame `watch-all` CPU under WSL (#356, reported by @Blainexi). `watch-all` pegged 15-30% CPU non-stop on WSL. NOT a busy loop: **`watchfiles` auto-enables polling whenever it detects WSL** (inotify unreliable across the boundary, confirmed in watchfiles docs), and its default 300ms poll re-stats the whole watched tree several times/sec — multiplied across every indexed repo. `DefaultFilter` drops `node_modules`/`.git` *events* but the polling walk still stats them (structural cost). Fix: the `awatch` call now passes `poll_delay_ms=_watch_poll_delay_ms()` (resolves `JCODEMUNCH_WATCH_POLL_DELAY_MS` → `WATCHFILES_POLL_DELAY_MS` → `DEFAULT_WATCH_POLL_DELAY_MS=1000`; non-positive/garbage → default); ignored when native FS events are used (no-op off WSL). Plus a one-time WSL startup hint (`_is_wsl()`) naming the two levers: raise the poll delay, or set `WATCHFILES_FORCE_POLLING=false` for Linux-filesystem repos to use native inotify (~0 idle CPU); `/mnt/*` repos need polling. NO INDEX_VERSION bump. New `tests/test_v1_108_83.py` (11); `test_watch_all.py`/`test_watcher_lock.py` green. Files: `watcher.py`, `watch_all.py`. See [[project_jmunch_console]].
45
- **Version:** 1.108.82 — Exclude `org_savings.db` from `list_repos`. The team-SKU org-rollup store (`org_savings.db`, written by `org/store.py`) sits in `~/.code-index/` next to per-repo index `.db` files, but `_NON_REPO_DB_FILES` only excluded `telemetry.db` → `list_repos` enumerated it as a phantom `local/org_savings` repo (sym 0, empty source_root, display==repo_id). In the jMunch Console it showed an un-removable card; `delete-index local/org_savings` failed (no matching `local-org_savings.db` index → non-zero exit), which the Console surfaced as the misleading "needs jcodemunch-mcp >= 1.108.50". Fix: `_NON_REPO_DB_FILES = frozenset({"telemetry.db", "org_savings.db"})` (1-line). NO INDEX_VERSION bump. New `tests/test_telemetry_db_skip.py::test_list_repos_skips_org_savings_db`; storage/sqlite_store suites green. Files: `storage/sqlite_store.py`. Console-side companion: the Console's delete-failure message now surfaces jcm's real error instead of the hard-coded version hint (jmunch-console). See [[project_jmunch_console]], [[project_org_rollup_sku]].
56
- **Version:** 1.108.81 — Lazy git identity probe in `get_watch_status` / `list-repos`. `list-repos` (and `get_watch_status`) slowed to 60s+ on many-repo hosts because the per-folder loop resolved each repo's reindex-state key up front via `_reindex_key` → `_local_repo_id` → `resolve_index_identity` — a **git identity probe ≈1 subprocess/repo** (~0.2s each here, worse in worktree-heavy trees; cf. the #303 resolve_repo perf history). But reindex_state is **in-memory only** (populated by a watcher/server process); a cold `list-repos` CLI (the jMunch Console cockpit shells exactly this) has none, so every probe resolved a key only to read an empty default. Fix: new `reindex_state.has_any_reindex_state()` (`bool(_repo_states)` under lock); `get_watch_status` computes `track_reindex = has_any_reindex_state()` ONCE before the loop and skips `_reindex_key`/`get_reindex_status` entirely when False, substituting the default `{index_stale:False, reindex_in_progress:False, stale_since_ms:None}` per repo. **Output shape unchanged** (cold path already returned these defaults — just after paying for N git probes); a watcher/server process WITH state still resolves keys so #353 crash-loop visibility holds. `process_locks.inspect` (OpenProcess, ~0.01s/10) and `service_status` (~0.26s) were measured cheap; the git probe was the sole per-repo scaler. NO INDEX_VERSION bump. New `tests/test_v1_108_81.py` (2: probe skipped + get_reindex_status not consulted when no state; keys resolved when state present); `test_v1_108_78.py`/`test_watch_all.py`/`test_reindex_state.py`/`test_watcher_lock.py` still green. Files: `reindex_state.py`, `tools/get_watch_status.py`. Motivated by the jMunch Console showing fixtures because its 20s CLI timeout lost to this; see [[project_jmunch_console]].
67
- **Version:** 1.108.80 — Dataclass/Pydantic field extraction in outlines (#355, @mmashwani; closes the last issue of the #351-355 batch). `get_file_outline` listed a dataclass as a bare `class` row → agents mistook it for the complete contract. Now **field-centric Python classes emit their fields as `kind="field"` child symbols.** Detection (`_is_field_centric_class` in `parser/extractor.py`): decorator final-name in {dataclass, attrs s/define/frozen/…} (handles call form `@dataclass(frozen=True)` + dotted `@pydantic.dataclasses.dataclass` via `_decorator_final_name`) OR a base class named `BaseModel`/`BaseSettings` (Pydantic, via `_node_final_name` on the class's `argument_list`). `_extract_python_class_fields` walks the class `block` for annotated `assignment` nodes (`left`/`type`/`right` field accessors), emits a `field` symbol per field (name, signature=full `name: type = default` text, line, parent=class id, content_hash). **`ClassVar[...]` skipped** (not a data field); **plain non-field-centric classes untouched** (no constant→field conflation — the reporter's explicit concern). Methods/class symbol unaffected, no dup. Fields flow into outline + `search_symbols(kind="field")` + `get_symbol_source`. Additive, NO INDEX_VERSION bump (re-index picks up fields). Also fixed `summarizer/file_summarize.py` heuristic counting every class child as a method → now `(N methods, M fields)`. Integration point: in `_walk_tree` after the class symbol is appended. New `tests/test_v1_108_80.py` (9: dataclass/frozen/pydantic/attrs extracted, ClassVar skipped, plain untouched, no method dup, outline integration, summary count). Full suite **4939 passed/7 skipped**. Files: `parser/extractor.py`, `summarizer/file_summarize.py`, `LANGUAGE_SUPPORT.md`. **mmashwani #351-355 batch now FULLY CLEARED.**
@@ -287,6 +288,7 @@ Tree-sitter grammar lacks clean named fields for these — custom regex extracto
287288
| `GEMINI_EMBED_TASK_AWARE` | 1 | Set `0`/`false`/`no`/`off` to disable task-type hints (`RETRIEVAL_DOCUMENT` / `CODE_RETRIEVAL_QUERY`) when using Gemini embeddings |
288289
| `JCODEMUNCH_CROSS_REPO_DEFAULT` | 0 | Set 1 to enable cross-repo traversal by default in find_importers, get_blast_radius, get_dependency_graph |
289290
| `JCODEMUNCH_EVENT_LOG` || Set `1` to write `_pulse.json` on every tool call (per-call activity signal for dashboards) |
291+
| `JCODEMUNCH_WATCH_POLL_DELAY_MS` | 1000 | (v1.108.83) Poll interval (ms) used ONLY when watchfiles falls back to polling — which it auto-enables under WSL (#356). Default raised from watchfiles' 300ms to cut idle CPU; ignored when native FS events are in use. Falls back to `WATCHFILES_POLL_DELAY_MS` if set; non-positive/garbage → default. For Linux-filesystem repos under WSL, `WATCHFILES_FORCE_POLLING=false` opts back into inotify (~0 idle CPU). |
290292
| `JCODEMUNCH_LIVE_JOURNAL` | 1 | (v1.108.57) Set `0`/`false`/`no`/`off` to disable the live session-journal write (`<CODE_INDEX_PATH>/_session_live.json`). On by default so the out-of-process PreCompact hook can read real session state (#334); throttled ≤1/~2s, paths+queries only, no file contents. |
291293
| `JCODEMUNCH_TOOL_SURFACE` | `full` | (v1.108.66) Tool surface selector (config key `tool_surface`; env wins). `counter` collapses `list_tools` to the 3-tool front door (`order`/`menu`/`route`) + always-present controls. Any other value (default `full`) preserves existing tiered behavior byte-for-byte — front-door tools stay hidden but callable. Composes with the `core`/`standard`/`full` tier profiles. |
292294
| `JCODEMUNCH_PARSE_CACHE` || Shared directory for the content-addressed parse cache (v1.108.40). Point all seats on a multi-home-dir box at the same path so identical files parse once across seats. Unset = disabled (no caching). |

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "jcodemunch-mcp"
3-
version = "1.108.82"
3+
version = "1.108.83"
44
description = "Token-efficient MCP server for source code exploration via tree-sitter AST parsing"
55
readme = "README.md"
66
requires-python = ">=3.10"

src/jcodemunch_mcp/watch_all.py

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,13 @@
2525
from typing import IO, Optional
2626

2727
from .storage import IndexStore
28-
from .watcher import DEFAULT_DEBOUNCE_MS, WatcherManager
28+
from .watcher import (
29+
DEFAULT_DEBOUNCE_MS,
30+
WatcherManager,
31+
_is_wsl,
32+
_watch_poll_delay_ms,
33+
_watcher_output,
34+
)
2935

3036
logger = logging.getLogger(__name__)
3137

@@ -96,6 +102,21 @@ async def watch_all(
96102
log_file_handle=log_file_handle,
97103
)
98104

105+
# WSL forces watchfiles into polling (inotify is unreliable across the
106+
# boundary), which re-stats every watched tree on an interval and can peg the
107+
# CPU on a many-repo host (#356). Surface the levers once at startup so the
108+
# user isn't left guessing why the daemon is busy.
109+
if _is_wsl():
110+
_watcher_output(
111+
"jcodemunch-mcp watch-all: WSL detected -> watchfiles is polling "
112+
f"(every {_watch_poll_delay_ms()}ms). To cut CPU: raise "
113+
"JCODEMUNCH_WATCH_POLL_DELAY_MS, or for repos on the Linux filesystem "
114+
"set WATCHFILES_FORCE_POLLING=false to use native inotify (near-zero "
115+
"idle CPU). Repos under /mnt/* need polling; moving them onto the "
116+
"Linux filesystem is faster all round.",
117+
quiet=quiet, log_file_handle=log_file_handle,
118+
)
119+
99120
stop_event = asyncio.Event()
100121
try:
101122
loop = asyncio.get_running_loop()

src/jcodemunch_mcp/watcher.py

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,45 @@
3131
# Default debounce in milliseconds
3232
DEFAULT_DEBOUNCE_MS = 200
3333

34+
# Poll interval (ms) used ONLY when watchfiles falls back to polling instead of
35+
# native FS events. watchfiles auto-enables polling whenever it detects WSL
36+
# (inotify is unreliable across the WSL boundary), and its own default of 300ms
37+
# re-stats the entire watched tree ~3x/second per repo — pegging the CPU on a
38+
# many-repo / large-tree host (issue #356). For a background *freshness* daemon a
39+
# ~1s cadence is invisible, so we raise the floor and let the user tune it.
40+
# Ignored entirely when native events are in use (every non-WSL Linux/mac/Win).
41+
DEFAULT_WATCH_POLL_DELAY_MS = 1000
42+
43+
44+
def _watch_poll_delay_ms() -> int:
45+
"""Resolve the polling interval, honoring JCODEMUNCH_WATCH_POLL_DELAY_MS
46+
(then watchfiles' own WATCHFILES_POLL_DELAY_MS as a fallback the user may
47+
already know), else DEFAULT_WATCH_POLL_DELAY_MS. Non-positive / unparseable
48+
values fall back to the default."""
49+
for var in ("JCODEMUNCH_WATCH_POLL_DELAY_MS", "WATCHFILES_POLL_DELAY_MS"):
50+
raw = os.environ.get(var)
51+
if raw is None:
52+
continue
53+
try:
54+
val = int(raw)
55+
except ValueError:
56+
continue
57+
if val > 0:
58+
return val
59+
return DEFAULT_WATCH_POLL_DELAY_MS
60+
61+
62+
def _is_wsl() -> bool:
63+
"""True when running under the Windows Subsystem for Linux (watchfiles polls
64+
here regardless of where the repo lives)."""
65+
if sys.platform != "linux":
66+
return False
67+
try:
68+
with open("/proc/version", "rt", encoding="utf-8", errors="ignore") as fh:
69+
return "microsoft" in fh.read().lower()
70+
except OSError:
71+
return False
72+
3473

3574
class WatcherError(Exception):
3675
"""Base exception for watcher errors that should not kill the embedding process."""
@@ -274,6 +313,9 @@ def _build_hash_cache() -> None:
274313
debounce=debounce_ms,
275314
recursive=True,
276315
step=200,
316+
# Only consulted when watchfiles polls (e.g. under WSL); a higher delay
317+
# there is the difference between idle and pegged CPU (#356).
318+
poll_delay_ms=_watch_poll_delay_ms(),
277319
):
278320
relevant = [
279321
(change_type, path)

tests/test_v1_108_83.py

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
"""v1.108.83 — watch-all CPU under WSL (issue #356).
2+
3+
watchfiles auto-enables polling under WSL; its default 300ms poll re-stats every
4+
watched tree several times a second, pegging the CPU on a many-repo host. We
5+
raise the poll floor (tunable) and emit a one-time WSL hint. These tests pin the
6+
poll-delay resolution + WSL detection helpers (the awatch wiring itself needs the
7+
optional `watchfiles` extra, so it's exercised by the live watcher suite).
8+
"""
9+
from __future__ import annotations
10+
11+
import builtins
12+
13+
import pytest
14+
15+
from jcodemunch_mcp import watcher
16+
17+
18+
# --- poll delay resolution ---------------------------------------------------
19+
20+
def _clear_delay_env(monkeypatch):
21+
monkeypatch.delenv("JCODEMUNCH_WATCH_POLL_DELAY_MS", raising=False)
22+
monkeypatch.delenv("WATCHFILES_POLL_DELAY_MS", raising=False)
23+
24+
25+
def test_default_poll_delay_is_raised_floor(monkeypatch):
26+
_clear_delay_env(monkeypatch)
27+
assert watcher._watch_poll_delay_ms() == watcher.DEFAULT_WATCH_POLL_DELAY_MS == 1000
28+
29+
30+
def test_jcm_env_overrides_poll_delay(monkeypatch):
31+
_clear_delay_env(monkeypatch)
32+
monkeypatch.setenv("JCODEMUNCH_WATCH_POLL_DELAY_MS", "2500")
33+
assert watcher._watch_poll_delay_ms() == 2500
34+
35+
36+
@pytest.mark.parametrize("bad", ["0", "-5", "junk", ""])
37+
def test_non_positive_or_garbage_falls_back_to_default(monkeypatch, bad):
38+
_clear_delay_env(monkeypatch)
39+
monkeypatch.setenv("JCODEMUNCH_WATCH_POLL_DELAY_MS", bad)
40+
assert watcher._watch_poll_delay_ms() == watcher.DEFAULT_WATCH_POLL_DELAY_MS
41+
42+
43+
def test_watchfiles_env_is_fallback(monkeypatch):
44+
_clear_delay_env(monkeypatch)
45+
monkeypatch.setenv("WATCHFILES_POLL_DELAY_MS", "1800")
46+
assert watcher._watch_poll_delay_ms() == 1800
47+
48+
49+
def test_jcm_env_takes_precedence_over_watchfiles_env(monkeypatch):
50+
_clear_delay_env(monkeypatch)
51+
monkeypatch.setenv("WATCHFILES_POLL_DELAY_MS", "1800")
52+
monkeypatch.setenv("JCODEMUNCH_WATCH_POLL_DELAY_MS", "4000")
53+
assert watcher._watch_poll_delay_ms() == 4000
54+
55+
56+
# --- WSL detection -----------------------------------------------------------
57+
58+
def test_is_wsl_false_on_non_linux(monkeypatch):
59+
monkeypatch.setattr(watcher.sys, "platform", "win32")
60+
assert watcher._is_wsl() is False
61+
62+
63+
def test_is_wsl_true_when_proc_version_names_microsoft(monkeypatch):
64+
monkeypatch.setattr(watcher.sys, "platform", "linux")
65+
real_open = builtins.open
66+
67+
def fake_open(path, *a, **k):
68+
if str(path) == "/proc/version":
69+
from io import StringIO
70+
return StringIO("Linux version 5.15.0-microsoft-standard-WSL2 ...")
71+
return real_open(path, *a, **k)
72+
73+
monkeypatch.setattr(builtins, "open", fake_open)
74+
assert watcher._is_wsl() is True
75+
76+
77+
def test_is_wsl_false_on_bare_metal_linux(monkeypatch):
78+
monkeypatch.setattr(watcher.sys, "platform", "linux")
79+
real_open = builtins.open
80+
81+
def fake_open(path, *a, **k):
82+
if str(path) == "/proc/version":
83+
from io import StringIO
84+
return StringIO("Linux version 6.8.0-generic (gcc ...) ")
85+
return real_open(path, *a, **k)
86+
87+
monkeypatch.setattr(builtins, "open", fake_open)
88+
assert watcher._is_wsl() is False

0 commit comments

Comments
 (0)