feat(scope): curated-CORE default - EVM-family flip + scope-friction observability (#733) - #830
Merged
Merged
Conversation
…observability (#733) An unconfigured install registered all 189 tools every turn because both scope levers defaulted to accept-all. Flip the defaults to the curated CORE set (ARCHITECTURE.md §6 R7): unset VAULTPILOT_CHAIN_FAMILIES now means {evm}, unset VAULTPILOT_PROTOCOLS now means the empty set — 71 tools registered by default, everything else opt-in. - parseFamilies(undefined) -> {evm}; parseProtocols(undefined) -> empty set. - Both env vars accept a first-class `all` token as the escape hatch back to the pre-flip surface (previously `all` only worked via the typo fallback). - Explicit values keep their prior behavior, typo fallback included. - Friction observability ships with the flip: getScopedOutFamilies / getScopedOutProtocols / getScopeEnableHint feed a new scope block on get_vaultpilot_config_status plus one startup stderr line.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #733
What changed and why
A default, unconfigured install registered all 189 tools every turn because both scope levers defaulted to accept-all (
parseFamilies(undefined)→ all five families,parseProtocols(undefined)→null). This flips the defaults to the curated-CORE set (ARCHITECTURE.md §6 R7, PROD option 2 on #721):src/config/scope.tsparseFamilies(undefined)now returns{evm}(newDEFAULT_FAMILIES, kept separate fromALL_FAMILIES).parseProtocols(undefined)now returns the empty set;nullsurvives only as the explicit accept-all sentinel.isProtocolEnabledis unchanged apart from a comment —null || has()still reads correctly under the new meaning.alltoken as the escape hatch back to the pre-flip surface. Before this PRVAULTPILOT_CHAIN_FAMILIES=all"worked" only by accident (it parsed as a typo and hit the all-families fallback); with a narrow default that escape hatch is load-bearing, so it is explicit now.getScopedOutFamilies()/getScopedOutProtocols()/getScopeEnableHint().src/modules/diagnostics/index.ts—get_vaultpilot_config_status'sscopeblock gainsscopedOutFamilies,scopedOutProtocols,enableHint.src/index.ts— one startup stderr line enumerating active vs scoped-out families/protocols + the enable hint, emitted only when something is actually scoped out.Friction measure (PROD-required, ships with the flip). The clean flip un-registers scoped-out tools, so the host never sees them and there is no in-band per-call refusal to count. Per the issue, the signal is the config-status scope block + the startup line, with
request_capabilityas the demand signal. The heavier register-but-refuse counter is not built here — the issue says not to without PROD/ARCH confirmation.Measured result: 71 tools registered by default (45 chain-agnostic core + 26 EVM-family), down from 189. R7's ≤72 bound holds with one tool of headroom.
Falsifier test
test/scope-tool-registration.test.ts(new). It walks everyregisterTool(server, "<name>", …)call site insrc/index.ts(same parse strategy astest/presign-annotation-guard-735.test.ts;main()is skipped under vitest so real SDK registrations can't be counted) and applies the productionisToolEnabledpredicate to each name. Every call site is unconditional at top level insidemain(), so "accepted byisToolEnabled" is exactly "registered".<= 72registered tools plus the exact 71-name pinned set. On unfixed code the default surface is all 189 — both assertions go red.getToolScope, not from the pinned list, so pasting a new snapshot can't launder a regression): nothing kept carries a non-EVM family or any protocol tag, and no chain-agnostic core tool was dropped (the fund-safety direction —send_transaction, contacts, Ledger verifiers must never fall out).VAULTPILOT_CHAIN_FAMILIES=all+VAULTPILOT_PROTOCOLS=allre-registers all 189.test/scope.test.tsgains the same falsifier at theisToolEnabledlevel, explicit-config parity tests, and coverage of the scoped-out accessors + hint.test/diagnostics-config-status.test.tsgains the config-status wiring assertions.There is no Node/npm on the authoring machine, so nothing here was executed — no build, no lint, no test run, and no claim of a green run. CI (Build & Test on Node 20 + 22) is the authoritative check. Tests were written by matching the existing files' framework, imports and mocking patterns (
vi.resetModules()+vi.stubEnv+ dynamic import, sincescope.tsreads its env vars once at module eval).Existing tests updated (behavior deliberately changed)
test/scope.test.ts—"default (no env vars) — every tool registered"→"default (no env vars) — EVM family + core only, no other family, no protocol". It asserted exactly the behavior this PR changes; it is now the falsifier for the new one, not a weakened assertion.test/scope.test.ts—"VAULTPILOT_PROTOCOLS unset → null (all protocols allowed)"→"… → empty set (protocols are opt-in)", with a new companion test pinningall→ thenullsentinel.test/scope.test.ts— two family-axis tests (CHAIN_FAMILIES=evm, the alias test) now also stubVAULTPILOT_PROTOCOLS=all. Their subject is the family axis; without the stub the new protocol default would gateprepare_aave_supplyand the tests would be asserting the wrong thing. No assertion was relaxed.No other existing assertion was touched. The #712 co-scoping guard and the #726 sunswap tag test are untouched and still green by construction (sunswap is
{family: tron}, so it is correctly scoped out by default).Blast radius
get_vaultpilot_config_statusresponse shape gains three fields (additive).scope.protocolschanges fromnullto[]for a default install.getToolScopetag changes, no handler/signing-path logic changes.Fund-safety / security note — NOT self-certified
This touches the registration gate in front of signing-class tools (
prepare_*,preview_*,send_transaction), so it is fund-safety-adjacent. It is not self-certified and needs maintainer security review. Specifically worth an independent look:preview_sendand the EVMprepare_*tools stay together (both{family: evm}), and the whole Solana/TRON/BTC/LTC surface leaves together, so no family ships aprepare_*without itspreview_*guard. fund-safety: regression test — preview_send/preview_solana_send must co-scope with their family's prepare/send tools #712's test asserts the tag relationship; the new registration test adds "no core tool dropped" from the other direction. A reviewer should confirm no other guard/gate pairing spans the family boundary.prepare_token_approveandread_contractare EVM-flavoured but fall throughgetToolScopeto the always-on core bucket. They stay enabled under the EVM default so this PR doesn't regress anything, but a user who setsVAULTPILOT_CHAIN_FAMILIES=solanastill gets both. Worth a follow-up issue; the issue's blast-radius section says no tag changes beyond scope: prepare_sunswap_swap (TRON DEX) falls through getToolScope → ships always-on, escapes family scoping #726 here.Residual concerns
parseFamilies(undefined)returns all five families (L73),parseProtocols(undefined)returns null (L87)") describes the pre-flip state, as does the R7 row's framing. I deliberately did not edit it — spec/architecture docs are the doc owner's (ARCH) surface, not something an implementation PR rewrites to match itself. Routing it as a finding: ARCH should refresh §6 and R7's status once this lands (note docs(arch): maintenance — R7/R8 settled, §8 issue numbers, review nits, cache-race reconcile #740 is already touching that doc).scripts/bench-tools.mjs(measurement: p95 latency bench for get/prepare/preview_send (resolves R1-R3 from UNMEASURED) #719) now benches the narrow default surface unless the operator setsVAULTPILOT_CHAIN_FAMILIES=all VAULTPILOT_PROTOCOLS=all. The R8 byte budget is not gated in this unit, but whoever runs that bench needs to know which surface they are measuring.DEFAULT_TOOL_SURFACEand fails the test. That is the point (scope.ts's own doc comment asks for it — a new tool that matches no rule silently lands in the always-on core bucket), but it is a maintenance cost worth an explicit ack.VAULTPILOT_PROTOCOLS=" , "now yields the empty set instead ofnull. Consistent with the flip; noting it since it is a behavior change on an explicitly-set (if degenerate) value.feat/733-curated-core-evm-flipalready existed on the remote (pushed earlier today under the same account, no PR attached) — apparently an interrupted earlier attempt at this same issue. I did not force-push or touch it; this PR is fromfeat/733-curated-core-evm-flip-v2. The orphan branch should be deleted by whoever owns it.