feat(attack-sim): AI-coordinated red-team campaign + 9 verified seeds#34
Merged
Conversation
…ategory breakdowns
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacks on #33 (base =
feat/attack-sim-harness, notmaster) so the diff is the campaign only and merges fast-forward. Retarget tomasteronce #33 lands.What this adds
An AI-coordinated red-team campaign that discovers detection bypasses and proposes them for human-gated inclusion in the Phase A corpus.
score_payloads.py(committed): batch scorer — arbitrary payloads → the realSusPatternsManager.detect(). The bridge that lets campaign agents (and humans) test candidates deterministically..claude/workflows/attack-campaign.workflow.js(untracked, not in this PR): the orchestration — depth+breadth attacker squads → scorer agent → default-reject verifiers → loop-until-dry → synthesize a staging report. Local by design; nothing AI-minted is committed without human approval.README-campaign.md(committed): the human-gate procedure (re-verify each proposal against the real detector before promoting).baseline.json(committed).Smoke result
One round on
haiku(cheap) generated 168 candidates → 21 proposed. Re-verifying the top 12 against the real detector confirmed all 12 are genuine misses. Distilled 9 into the corpus (SQLi DDL / ORDER BY / MySQL version-comment, ERB + OGNL SSTI, netcat + fullwidth-;cmd injection, JSON-quoted NoSQL operators).Baseline moves 0.658 → 0.421 — expected and honest: we added real, currently-undetected attacks, and the ratchet now tracks them.
Notable
The campaign also flagged a likely
ContentPreprocessornormalization-order bug (comment stripping mergesSELECT/**/FROM→SELECTFROM). That's a code fix, queued as the first follow-up item — not addressed here.Scope / safety
Test-only additions under
tests/attack_simulation/plus the regenerated baseline. No shippedguard_core/code changed. Thesync-mirror --checkCI may be red for pre-existing reasons unrelated to this change.