Skip to content

Commit c118647

Browse files
affaan-maffaan-m
committed
test: add vitest suite with 82 tests across 7 files
- tests/rules/secrets.test.ts (14 tests) - tests/rules/permissions.test.ts (13 tests) - tests/rules/hooks.test.ts (11 tests) - tests/rules/mcp.test.ts (13 tests) - tests/rules/agents.test.ts (12 tests) - tests/scanner/scanner.test.ts (9 tests) - tests/reporter/score.test.ts (10 tests) Also adds CLAUDE.md with project context and vitest.config.ts.
1 parent bb42845 commit c118647

10 files changed

Lines changed: 1018 additions & 0 deletions

File tree

‎CLAUDE.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# AgentShield
2+
3+
Security auditor for AI agent configurations (Claude Code, MCP servers, hooks, agents).
4+
5+
## Build & Test
6+
7+
```bash
8+
npm run build # tsc + tsup → dist/
9+
npm test # vitest (82 tests)
10+
npm run dev # tsx watch mode
11+
```
12+
13+
## Architecture
14+
15+
```
16+
src/
17+
index.ts # CLI entry (commander)
18+
types.ts # Core types + Zod schemas
19+
scanner/
20+
discovery.ts # File discovery (CLAUDE.md, settings.json, mcp.json, agents/, etc.)
21+
index.ts # Orchestrates discovery → rules → sorted findings
22+
rules/
23+
index.ts # Barrel export of all rule modules
24+
secrets.ts # Hardcoded API keys, tokens, passwords (11 patterns)
25+
permissions.ts # Allow/deny list analysis, dangerous flags
26+
hooks.ts # Injection, exfiltration, silent error suppression
27+
mcp.ts # Risky servers, hardcoded env, npx supply chain
28+
agents.ts # Tool restrictions, prompt injection surface, CLAUDE.md injection
29+
reporter/
30+
score.ts # Scoring engine (severity deductions, grade A-F, category breakdown)
31+
terminal.ts # Colored terminal output
32+
json.ts # JSON + Markdown report formats
33+
index.ts # Format dispatcher
34+
opus/
35+
prompts.ts # System prompts for Attacker/Defender/Auditor
36+
pipeline.ts # Opus 4.6 three-agent adversarial pipeline
37+
render.ts # Opus analysis terminal + markdown rendering
38+
index.ts # Pipeline entry point
39+
```
40+
41+
## Key Patterns
42+
43+
- **Rules**: Each rule module exports `ReadonlyArray<Rule>`. Each `Rule` has `check(file: ConfigFile): ReadonlyArray<Finding>`.
44+
- **Immutability**: All arrays typed as `ReadonlyArray`, all interfaces use `readonly` fields.
45+
- **No RegExp .prototype methods**: Use `String.matchAll()` via `findAllMatches()` helper to avoid security hook conflicts.
46+
- **False positive prevention**: `parsePermissionLists()` JSON-parses settings to check only the allow array. Negation-aware context checking downgrades prohibitive mentions to `info`.
47+
48+
## Severity Scoring
49+
50+
| Severity | Deduction | Example |
51+
|----------|-----------|---------|
52+
| critical | -25 | Hardcoded API key, Bash(*) |
53+
| high | -15 | Shell MCP server, no deny list |
54+
| medium | -5 | Unrestricted curl, missing denials |
55+
| low | -2 | No model specified in agent |
56+
| info | 0 | Missing description, good practice |
57+
58+
Grades: A (>=90), B (>=75), C (>=60), D (>=40), F (<40)
59+
60+
## CLI
61+
62+
```bash
63+
agentshield scan [path] # Static analysis
64+
agentshield scan --opus # + Opus 4.6 adversarial pipeline
65+
agentshield scan --format json|md # Output format
66+
agentshield scan --fix # Show auto-fix suggestions
67+
```
68+
69+
## Testing
70+
71+
Tests in `tests/` mirror `src/` structure. Use `makeFinding()`, `makeSettings()`, etc. helper factories.
72+
Run specific suite: `npx vitest run tests/rules/mcp.test.ts`
73+
74+
## Conventions
75+
76+
- TypeScript strict mode, ESM modules
77+
- No mutation, no `any`, no `console.log` in src
78+
- Zod for config validation at boundaries
79+
- Conventional commits: feat/fix/test/refactor/docs

‎CONTRIBUTING.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# Contributing to AgentShield
2+
3+
## Setup
4+
5+
```bash
6+
git clone https://github.com/affaan-m/agentshield.git
7+
cd agentshield
8+
npm install
9+
npm run typecheck
10+
npm run scan:demo # verify everything works
11+
```
12+
13+
Requires Node >= 20.
14+
15+
## Development
16+
17+
```bash
18+
npm run dev scan --path examples/vulnerable # run scanner in dev mode
19+
npm run build # build with tsup
20+
npm run typecheck # type check
21+
npm test # run tests
22+
npm run test:coverage # tests with coverage
23+
```
24+
25+
## Running Tests
26+
27+
Tests use [Vitest](https://vitest.dev) and live in `tests/`. Convention:
28+
29+
```
30+
tests/
31+
├── rules/
32+
│ ├── secrets.test.ts
33+
│ ├── mcp.test.ts
34+
│ ├── hooks.test.ts
35+
│ ├── permissions.test.ts
36+
│ └── agents.test.ts
37+
├── scanner/
38+
│ └── discovery.test.ts
39+
└── reporter/
40+
└── score.test.ts
41+
```
42+
43+
Each test file mirrors its source counterpart. Run a single file with:
44+
45+
```bash
46+
npx vitest tests/rules/secrets.test.ts
47+
```
48+
49+
## Adding a New Rule
50+
51+
Rules live in `src/rules/`. Each rule implements the `Rule` interface from `src/types.ts`:
52+
53+
```typescript
54+
import type { ConfigFile, Finding, Rule } from "../types.js";
55+
56+
export const myRules: ReadonlyArray<Rule> = [
57+
{
58+
id: "category-short-name",
59+
name: "Human-Readable Name",
60+
description: "What this rule checks for",
61+
severity: "high", // critical | high | medium | low | info
62+
category: "permissions", // secrets | permissions | hooks | mcp | agents | injection | exposure | misconfiguration
63+
check(file: ConfigFile): ReadonlyArray<Finding> {
64+
const findings: Finding[] = [];
65+
66+
// Your detection logic here.
67+
// file.content is the raw file text.
68+
// file.type tells you what kind of config it is.
69+
// Return findings with evidence and optional fix suggestions.
70+
71+
return findings;
72+
},
73+
},
74+
];
75+
```
76+
77+
Then register your rules in `src/rules/index.ts`:
78+
79+
```typescript
80+
import { myRules } from "./my-rules.js";
81+
82+
export function getBuiltinRules(): ReadonlyArray<Rule> {
83+
return [
84+
...secretRules,
85+
...permissionRules,
86+
...hookRules,
87+
...mcpRules,
88+
...agentRules,
89+
...myRules, // add here
90+
];
91+
}
92+
```
93+
94+
Add a vulnerable example in `examples/vulnerable/` that triggers your rule, and write a test in `tests/rules/`.
95+
96+
## Providing Auto-Fix Suggestions
97+
98+
If your rule can suggest a fix, include a `Fix` object:
99+
100+
```typescript
101+
fix: {
102+
description: "What the fix does",
103+
before: "the problematic text",
104+
after: "the safe replacement",
105+
auto: true, // true = safe to apply with --fix, false = manual review needed
106+
}
107+
```
108+
109+
Only set `auto: true` if the fix is safe to apply without human review.
110+
111+
## Code Style
112+
113+
- Immutable by default — use `readonly` on all interface fields and `ReadonlyArray`
114+
- Pure functions where possible — rules are `(ConfigFile) => Finding[]`
115+
- No mutation — create new objects, don't modify existing ones
116+
- TypeScript strict mode — no `any`, no implicit returns
117+
118+
## Pull Requests
119+
120+
1. Fork the repo and create a branch from `main`
121+
2. Add your rule + test + vulnerable example
122+
3. Run `npm run typecheck && npm test && npm run scan:demo`
123+
4. Open a PR with a clear description of what the rule catches and why it matters
124+
125+
## License
126+
127+
By contributing, you agree that your contributions will be licensed under the MIT License.

‎tests/reporter/score.test.ts‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
import { describe, it, expect } from "vitest";
2+
import { calculateScore } from "../../src/reporter/score.js";
3+
import type { Finding, ScanTarget } from "../../src/types.js";
4+
import type { ScanResult } from "../../src/scanner/index.js";
5+
6+
function makeScanResult(findings: Finding[]): ScanResult {
7+
const target: ScanTarget = {
8+
path: "/test",
9+
files: [{ path: "test.json", type: "settings-json", content: "{}" }],
10+
};
11+
return { target, findings };
12+
}
13+
14+
function makeFinding(overrides: Partial<Finding> = {}): Finding {
15+
return {
16+
id: "test-finding",
17+
severity: "medium",
18+
category: "permissions",
19+
title: "Test finding",
20+
description: "Test description",
21+
file: "test.json",
22+
...overrides,
23+
};
24+
}
25+
26+
describe("calculateScore", () => {
27+
it("returns Grade A for no findings", () => {
28+
const result = makeScanResult([]);
29+
const report = calculateScore(result);
30+
expect(report.score.grade).toBe("A");
31+
expect(report.score.numericScore).toBe(100);
32+
});
33+
34+
it("deducts 25 points per critical finding", () => {
35+
const result = makeScanResult([
36+
makeFinding({ id: "c1", severity: "critical" }),
37+
]);
38+
const report = calculateScore(result);
39+
expect(report.score.numericScore).toBe(75);
40+
expect(report.score.grade).toBe("B");
41+
});
42+
43+
it("deducts 15 points per high finding", () => {
44+
const result = makeScanResult([
45+
makeFinding({ id: "h1", severity: "high" }),
46+
]);
47+
const report = calculateScore(result);
48+
expect(report.score.numericScore).toBe(85);
49+
expect(report.score.grade).toBe("B");
50+
});
51+
52+
it("deducts 5 points per medium finding", () => {
53+
const result = makeScanResult([
54+
makeFinding({ id: "m1", severity: "medium" }),
55+
makeFinding({ id: "m2", severity: "medium" }),
56+
]);
57+
const report = calculateScore(result);
58+
expect(report.score.numericScore).toBe(90);
59+
});
60+
61+
it("floors score at 0", () => {
62+
const result = makeScanResult([
63+
makeFinding({ id: "c1", severity: "critical" }),
64+
makeFinding({ id: "c2", severity: "critical" }),
65+
makeFinding({ id: "c3", severity: "critical" }),
66+
makeFinding({ id: "c4", severity: "critical" }),
67+
makeFinding({ id: "c5", severity: "critical" }),
68+
]);
69+
const report = calculateScore(result);
70+
expect(report.score.numericScore).toBe(0);
71+
expect(report.score.grade).toBe("F");
72+
});
73+
74+
it("does not deduct for info findings", () => {
75+
const result = makeScanResult([
76+
makeFinding({ id: "i1", severity: "info" }),
77+
makeFinding({ id: "i2", severity: "info" }),
78+
]);
79+
const report = calculateScore(result);
80+
expect(report.score.numericScore).toBe(100);
81+
});
82+
83+
it("correctly counts findings by severity in summary", () => {
84+
const result = makeScanResult([
85+
makeFinding({ id: "c1", severity: "critical" }),
86+
makeFinding({ id: "h1", severity: "high" }),
87+
makeFinding({ id: "h2", severity: "high" }),
88+
makeFinding({ id: "m1", severity: "medium" }),
89+
makeFinding({ id: "l1", severity: "low" }),
90+
makeFinding({ id: "i1", severity: "info" }),
91+
]);
92+
const report = calculateScore(result);
93+
expect(report.summary.critical).toBe(1);
94+
expect(report.summary.high).toBe(2);
95+
expect(report.summary.medium).toBe(1);
96+
expect(report.summary.low).toBe(1);
97+
expect(report.summary.info).toBe(1);
98+
expect(report.summary.totalFindings).toBe(6);
99+
});
100+
101+
it("counts auto-fixable findings", () => {
102+
const result = makeScanResult([
103+
makeFinding({ id: "f1", fix: { description: "fix", before: "a", after: "b", auto: true } }),
104+
makeFinding({ id: "f2", fix: { description: "fix", before: "a", after: "b", auto: false } }),
105+
makeFinding({ id: "f3" }),
106+
]);
107+
const report = calculateScore(result);
108+
expect(report.summary.autoFixable).toBe(1);
109+
});
110+
111+
it("maps categories to score breakdown correctly", () => {
112+
const result = makeScanResult([
113+
makeFinding({ id: "s1", severity: "critical", category: "secrets" }),
114+
makeFinding({ id: "m1", severity: "high", category: "mcp" }),
115+
makeFinding({ id: "a1", severity: "medium", category: "agents" }),
116+
]);
117+
const report = calculateScore(result);
118+
expect(report.score.breakdown.secrets).toBe(75); // 100 - 25
119+
expect(report.score.breakdown.mcp).toBe(85); // 100 - 15
120+
expect(report.score.breakdown.agents).toBe(95); // 100 - 5
121+
expect(report.score.breakdown.permissions).toBe(100); // untouched
122+
expect(report.score.breakdown.hooks).toBe(100); // untouched
123+
});
124+
125+
it("grades correctly at boundaries", () => {
126+
// A: >= 90
127+
expect(calculateScore(makeScanResult([
128+
makeFinding({ id: "m1", severity: "medium" }),
129+
makeFinding({ id: "m2", severity: "medium" }),
130+
])).score.grade).toBe("A");
131+
132+
// B: 75-89
133+
expect(calculateScore(makeScanResult([
134+
makeFinding({ id: "c1", severity: "critical" }),
135+
])).score.grade).toBe("B");
136+
137+
// C: 60-74
138+
expect(calculateScore(makeScanResult([
139+
makeFinding({ id: "c1", severity: "critical" }),
140+
makeFinding({ id: "h1", severity: "high" }),
141+
])).score.grade).toBe("C");
142+
143+
// D: 40-59
144+
expect(calculateScore(makeScanResult([
145+
makeFinding({ id: "c1", severity: "critical" }),
146+
makeFinding({ id: "c2", severity: "critical" }),
147+
])).score.grade).toBe("D");
148+
149+
// F: < 40
150+
expect(calculateScore(makeScanResult([
151+
makeFinding({ id: "c1", severity: "critical" }),
152+
makeFinding({ id: "c2", severity: "critical" }),
153+
makeFinding({ id: "c3", severity: "critical" }),
154+
])).score.grade).toBe("F");
155+
});
156+
});

0 commit comments

Comments
 (0)