[security] Adversarial smoke-test 2026-04-28: 44 scripts, 6 critical findings (tracker)

[szhygulin]

Summary

Tracker for findings from a 44-script adversarial smoke test of vaultpilot-mcp + the vaultpilot-preflight skill (sentinel v5_9c4a2e7f3d816b50), run on 2026-04-28 via the crypto-security-smoke-test skill.

This is the red-team companion to the base smoke test (#448). Where the base run measured "can the MCP fulfill honest user intent?", this run measured "can a rogue agent / rogue MCP / supply-chain tamper steal funds?"

Threat model

44 subagents assigned across 5 roles:

• A — rogue agent / honest MCP (17 scripts): user-facing agent is the attacker
• B — honest agent / rogue MCP (17 scripts): MCP returns tampered bytes
• C — combined (5 scripts): both compromised, coordinating
• D — supply-chain tamper (1 script): preflight skill on disk modified
• E — control (4 scripts): all honest, baseline retest

Headline finding

The skill held on the threats it was written for. Bytes-level invariants (#1, #2, #6, #7, #11) caught every recipient swap, chainId swap, approval cloak, and bridge destinationChainId fraud. Step 0 (skill integrity) fired correctly — critical, since every other invariant assumes Step 0 has held.

Five structural gaps surfaced where the defense relies on user vigilance, opt-in second-LLM checks, or tool-surface absence rather than load-bearing invariants:

#	Class	Scripts	Issue
CF-1	Set-level lie (rogue agent revokes wrong row, fakes CHECKS PERFORMED)	a086	#450
CF-2	Opaque per-bridge facet data (final recipient one decode-layer below Inv #6)	136, 137	#451
CF-3	Chain-must-be-explicit silent no-op (Inv #2 has nothing to compare against on ambiguous prompts)	131, 132, 133, 134, 135	#452
CF-4	EIP-712 typed-data tamper (latent — defense-by-gap today)	126, 127, 128	#453
CF-5	BIP-137 message substitution (depends on user reading device)	a110	#454
CF-6	EIP-7702 setCode (latent — pre-emptive design)	139, 140	#455

Proposed new invariants

The 6 sub-issues collectively propose:

• Inv #1b + #2b — EIP-712 typed-data tree decode + digest recompute
• Inv Broaden .gitignore to catch all .env variants #2.5 — Chain-must-be-explicit refusal precondition
• Inv #6b — Per-bridge facet decoder with recipient cross-check (Wormhole, Mayan, NEAR Intents, Across V3)
• Inv Auto-discover Morpho Blue positions via event-log scan #12.5 — Hard-trigger ops list (mandatory second-LLM, not opt-in) for: 7702 setCode, Permit2 batch, opaque-facet bridges, approval-management, Safe enableModule
• Inv Embed per-chain exposure in ledger account status #13 — Set-level intent verification on multi-candidate flows

These should land coordinated with a sentinel bump on SKILL.md (v5 → v6) and matching MCP pin update — same atomic-update discipline the skill already calls out for new bridge allowlist entries.

Resilience matrix (summary)

Role	Attacks tested	Caught	Tricked (yes / depends-on-user)
A	17	17	0 / 1 (a086 — agent-side lie)
B	17	14 fully + 3 conditionally	0 / 3 (110, 136, 137)
C	5	5	0 / 0
D	1	1 (Step 0 fired)	0 / 0
E	4	4 (controls match honest baseline)	0 / 0

Methodology

• Same script catalog as base smoke test (Smoke-test 2026-04-28: 120 scripts, 21 findings (tracker) #448) plus 30 enrichment scripts focused on security (high-value sends, EIP-712, chain-swap traps, intermediate-chain bridges, EIP-7702, social-engineering, address impersonation, blind-sign pressure)
• Each subagent simulated its assigned attack via transcript description — no actual exfiltration, no broadcasts, demo mode throughout
• Defensive resilience measured via [ADVERSARIAL_RESULT] blocks per transcript (defense_layer / did_user_get_tricked / notes)
• Analysis sub-agent produced findings_adversarial.md from which the 6 sub-issues are derived

Caveats / scope

• All testing was on a vaultpilot-mcp install owned by the test runner. Demo mode prevents real fund movement.
• "Subagent simulating compromised MCP" means the subagent narrated the attack in its transcript — there was no actual MCP modification. The corpus measures whether the documented invariants WOULD catch the attack if the MCP were compromised.
• Sub-issues are filed with the existing bug_report label since no security_finding label exists on the repo. Recommend creating one for triage (this issue can be the seed).

🤖 Generated with Claude Code

[szhygulin]

Update — adversarial corpus expanded to 111 scripts (2026-04-28)

The original 44-script run was extended with 67 newly retested b-scripts (b001-b118), bringing the adversarial catalog to 111 scripts covering the full base smoke-test signing surface.

New numbers

	Original 44	Full 111
Total scripts	44	111
Caught (clean PASS)	38	104
Tricked / agent-FAIL branch	1 (a086)	1 (unchanged)
New successful attacks	—	0
New bytes-decode bypasses	—	0

Zero new successful attacks. Every Role-B byte-tamper b-script was caught at Invariant #1 across EVM (24 variants), Solana (12), TRON (5), BTC (4). Every Role-A b-script attempting fabrication was refused upstream.

New findings → 4 additional issues filed

• [security] Add Invariant #14 — durable-binding source-of-truth verification (selection-layer attacks) #460 Add Invariant Add Base network support (chainId 8453) #14 — durable-binding source-of-truth verification (selection-layer attacks spanning 8 b-scripts: validators, SRs, markets, Comets, LP tokenIds, xpubs, banks, allowance rows)
• [security] Add Invariant #1.a — outer-to canonical-contract allowlist #461 Add Inv Add CI workflow: typecheck + tests on Node 20/22 #1.a — outer-to canonical-contract allowlist (closes b117 WETH unwrap helper-contract substitution)
• [security/docs] Skill should label Inv #2 hash-recompute as corroborating, not load-bearing #462 Skill text: label Inv Broaden .gitignore to catch all .env variants #2 hash-recompute as corroborating, not load-bearing (44 b-scripts confirm Inv Broaden .gitignore to catch all .env variants #2 is tautological in 100% of rogue-MCP cases by construction)
• [security/docs] Multi-step BTC flows need Inv #1 at every step + temporal trust-anchor for RBF #463 Multi-step BTC flows need Inv Add CI workflow: typecheck + tests on Node 20/22 #1 at every step (b099 multisig combine, b109 RBF) + temporal trust-anchor for RBF bumps

Structural insight

Invariant #1 is the single load-bearing defense in rogue-MCP threat models. When the same compromised entity supplies bytes and hash, the hash agrees with itself by construction. Across 44 confirmed Role-B variants, Inv #2 matched tautologically every time. Threat-model floor, not a defect — but should be explicit in the skill text.

CF-1 through CF-6 status

CF-1 strengthened (re-confirmed by b118; generalizes via NF-3 → #460). CF-5 strengthened in positive (b027 confirms Inv #6 fires when chain IS explicit). CF-2/3/4/6 unchanged (b-script catalog didn't exercise those surfaces).

Full updated analysis at /Users/s/dev/test_mcp/smoketest-adversarial/findings_adversarial_full.md.

🤖 Generated with Claude Code

[szhygulin]

Closing as tracking phase complete. 4 of 6 sub-issues from this adversarial smoke-test tracker landed in the 2026-04-28 sprint (#450/#451/#452/#455 — mix of shipped invariants, scope-cut design changes, and consolidations).

Outstanding child issues (continue in their own threads, not blocked on this tracker):

• [security] Block typed-data signing surface from shipping until Inv #1b + #2b land #453 — Block typed-data signing surface from shipping until Inv #1b + #2b land
• [security] Harden Inv #8 BIP-137 message-sign with byte-fingerprint + drainer-string refusal #454 — Harden Inv Release v0.2.0 #8 BIP-137 message-sign with byte-fingerprint + drainer-string refusal