Commit 1ff7c60
release: v1.108.171 — provider discovery no longer scans in quadratic time
Root cause of #375, found by the reporter's mid-stall py-spy dump. Every sample
sat in ExpressProvider._extract_mounts, reached via discover_providers ->
_resolve_active_providers. That runs BEFORE the file walk, so a single large
file hung the entire index at 100% CPU with no progress notification (the
reporter's stdio client sends no progressToken) and no log line (index_folder
logs below the default WARNING level). 4m0s with context providers on, 5.1s
with them off.
The mechanism is a leading uncaptured (?:\w+) before the \. It asserts nothing
the . does not already imply, but it makes the scan QUADRATIC: at every offset
inside an unbroken word run the engine matches the run greedily, fails on the .,
then backtracks one character at a time. One minified chunk or base64 blob in a
single source file is enough.
py-spy named ONE pattern. Screening every compiled regex in the context
providers empirically, rather than reading them, found SIX across two providers:
_JS_ROUTE, _JS_MIDDLEWARE, _JS_MOUNT, _GO_ROUTE, _GO_MIDDLEWARE, _GO_GROUP. All
six showed ~16x time for 4x input. Fixing only the reported one would have left
five live.
Measured on a lone \w run, before -> after:
8k 0.45s -> 0.00001s
32k 7.48s -> 0.00002s
128k 120s+ -> 0.00006s
Every provider regex in the package now scans a 128k run in 0.0104s total.
Extraction is unchanged (the group was uncaptured; the . still requires a
receiver) and marginally more permissive, now also matching getRouter().use(...).
tests/test_v1_108_171.py (11). The scaling guard is EMPIRICAL rather than a
pattern-text assertion, because a future provider can reintroduce the blowup
with different syntax and only timing catches that; plus a non-vacuity check and
a named pin on the six so a careless revert fails with a clear message instead
of a stopwatch.
Two notes on verifying this, both learned the hard way here:
- The first non-vacuity probe was a shell one-liner whose escaping tangled, so
the assert fired before any write and the suite reported green against an
UNMODIFIED file. A probe that proves nothing looks exactly like a probe that
passes. Redone as a script file.
- The guard's blob was originally 128k, which meant a real regression took over
120 seconds to fail. A guard that hangs CI is worse than no guard. At 32k the
separation is still three orders of magnitude and it fails in 11.6s.
NOT included: the reporter's per-provider time budget. It removes the class
rather than this instance, but silently skipping a provider needs a disclosure
design, not a quick patch.
Suite 5716 passed + the known 12 local-ONNX semantic env failures.
NO schema, tool-count, or INDEX_VERSION change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent b02015e commit 1ff7c60
6 files changed
Lines changed: 253 additions & 164 deletions
File tree
- src/jcodemunch_mcp/parser/context
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
5 | 45 | | |
6 | 46 | | |
7 | 47 | | |
| |||
0 commit comments