|
1 | | -# Everything Claude Code — Lean |
| 1 | +# agent-trials |
2 | 2 |
|
3 | | -[](https://github.com/OsamaA140/everything-claude-code-lean/actions/workflows/tests.yml) |
| 3 | +[](https://github.com/OsamaA140/agent-trials/actions/workflows/tests.yml) |
4 | 4 |
|
5 | | -Production-ready agents, skills, hooks, commands, and rules for Claude Code — a **token-optimized fork** of [affaan-m/everything-claude-code](https://github.com/affaan-m/everything-claude-code) that keeps the coverage and cuts the cost. |
| 5 | +**Prove an agent does its job before you trust it.** Hire-ready employee templates, a deterministic trial harness that grades agents against fixtures with planted mistakes, zero-token safety hooks, and a token-lean agent/skill set for Claude Code. |
6 | 6 |
|
7 | | -## Why this fork: it barely costs tokens |
| 7 | +## Trials: behavioural testing for agents |
8 | 8 |
|
9 | | -Most plugin stacks quietly tax every session and every invocation. This one doesn't: |
| 9 | +Everyone ships agents. Nobody proves they work. A **trial** hands an employee a fake business with mistakes planted in it and checks deterministically whether it caught them, stayed inside its write scope, and got the arithmetic right. |
10 | 10 |
|
11 | | -Measured, not claimed — full methodology and reproduction steps in [docs/BENCHMARK.md](docs/BENCHMARK.md): |
| 11 | +```bash |
| 12 | +node scripts/trial.js prepare trials/ops-manager-smb --workspace /tmp/t1 |
| 13 | +node scripts/trial.js report trials/ops-manager-smb --runs /tmp/t1,/tmp/t2,/tmp/t3 |
| 14 | +``` |
| 15 | + |
| 16 | +The headline metric is **pass^k** — passed in *every* run, not merely once — because one success is capability, not reliability. No LLM judge is involved: [judge style bias runs 0.76-0.92](https://arxiv.org/abs/2604.23178), which grades prose instead of correctness. Three fixtures ship, including two held-out businesses used to separate judgement from memorisation. Methodology, measured results, and a retracted finding in [docs/TRIALS.md](docs/TRIALS.md). |
| 17 | + |
| 18 | +## Token cost: measured, not claimed |
12 | 19 |
|
13 | | -| | This fork | Upstream ECC (measured @ e4e4163) | |
| 20 | +Most stacks quietly tax every session and every invocation. Full methodology and reproduction steps in [docs/BENCHMARK.md](docs/BENCHMARK.md): |
| 21 | + |
| 22 | +| | agent-trials | A full-size stack (measured) | |
14 | 23 | |---|---|---| |
15 | 24 | | Always-on context cost | **~1,260 tokens** for all 26 skills + 9 agents | ~93,000 tokens for a full checkout | |
16 | 25 | | Avg agent invocation | **623 tokens** | 1,605 tokens (**61% less**) | |
17 | 26 | | Avg skill invocation | **455 tokens** | 2,203 tokens (**79% less**) | |
18 | 27 | | Avg command invocation | **189 tokens** | 970 tokens (**81% less**) | |
19 | 28 | | Hooks (secret blocking, command guard, formatting) | **0 tokens** — they run in the harness, not the model | 0 (same mechanism) | |
20 | | -| Verification | 100/100 unit tests, CI on Node 20/22, guards live-fired in real sessions | — | |
| 29 | +| Verification | 134 unit tests, CI on Node 20/22, guards live-fired in real sessions | — | |
21 | 30 |
|
22 | | -(Fair-comparison note: upstream is a marketplace meant to be partially enabled, and covers far more ground — the per-invocation averages are the like-for-like numbers. Details and caveats in the benchmark doc.) |
| 31 | +(Fair-comparison note: the comparison stack is a marketplace meant to be partially enabled and covers far more ground, so the per-invocation averages are the like-for-like numbers. Details and caveats in the benchmark doc.) |
23 | 32 |
|
24 | 33 | How: agents/skills state *when and why*, not re-teach *how* (Claude already knows the syntax); commands are thin pointers to agents instead of duplicating them; execution agents run on Sonnet with Opus reserved for architecture, planning, and security; and all enforcement lives in zero-token hooks. Full change log in `OPTIMIZATION.md` and `CHANGELOG.md`. |
25 | 34 |
|
@@ -52,8 +61,8 @@ mcp-configs/ Example MCP server configs (replace YOUR_*_HERE placeholders) |
52 | 61 |
|
53 | 62 | **As a plugin (recommended):** |
54 | 63 | ```bash |
55 | | -/plugin marketplace add <path-or-repo-you-host-this-at> |
56 | | -/plugin install everything-claude-code@everything-claude-code |
| 64 | +/plugin marketplace add OsamaA140/agent-trials |
| 65 | +/plugin install agent-trials@agent-trials |
57 | 66 | ``` |
58 | 67 |
|
59 | 68 | **Manual:** |
@@ -100,5 +109,8 @@ Don't enable every MCP at once — budget ~20-30 configured, <10 enabled per pro |
100 | 109 | node tests/run-all.js |
101 | 110 | ``` |
102 | 111 |
|
103 | | -## Credit & License |
104 | | -MIT, upstream by [Affaan Mustafa](https://x.com/affaanmustafa) — [original repo](https://github.com/affaan-m/everything-claude-code) (10+ months of production use, Anthropic x Forum Ventures hackathon winner). This fork only compresses/restructures for token efficiency; all credit for the underlying content and design goes upstream. Use freely, modify as needed. |
| 112 | +## License & Notice |
| 113 | + |
| 114 | +MIT — see [LICENSE](LICENSE). Use freely, modify as needed. |
| 115 | + |
| 116 | +The trial harness, guards library, agent validator, benchmark, employee templates, fixtures, and test suite are original to this project. Some agent, skill, command, and rule definitions, and parts of the cross-platform hook scripts, derive from [everything-claude-code](https://github.com/affaan-m/ECC) by Affaan Mustafa, used under the MIT License. |
0 commit comments