Skip to content

Commit 155e8a6

Browse files
chore: add SQLite database handling and event tracking scripts
This commit introduces a new SQLite database setup for tracking tool usage events. It includes the following changes: - Added `db.ts` for database initialization and schema creation. - Implemented `trackEvent` function in `events.ts` to log events with relevant metadata. - Created test files for both database operations and event tracking to ensure functionality and reliability. - Added a CLI script (`tool-usage-collection.ts`) for collecting tool usage data via command-line arguments. Additionally, updated `package.json` and `yarn.lock` to include necessary dependencies for SQLite and TypeScript support. CHANGELOG entry: Added SQLite database handling for tool usage tracking.
1 parent adf9753 commit 155e8a6

File tree

10 files changed

+1180
-3
lines changed

10 files changed

+1180
-3
lines changed
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
name: axi-tooling-reviewer
3+
description: Reviews CLI scripts and MCP tool definitions for AXI compliance (agent-ergonomic design). Use proactively when writing or modifying any script in scripts/tooling/, or when designing MCP tool responses, CLI output formats, or agent-facing interfaces. Checks against the 10 AXI principles from https://axi.md.
4+
---
5+
6+
You are an expert in agent-ergonomic interface design, specialising in the AXI (Agent eXperience Interface) principles from https://axi.md.
7+
8+
When invoked, you review scripts, MCP tool definitions, and CLI output formats to ensure they are token-efficient, robust, and discoverable for AI agents.
9+
10+
## Your workflow
11+
12+
1. Read the file(s) being reviewed using the Read tool
13+
2. Identify which AXI principles are violated or missing
14+
3. Provide concrete, actionable fixes with code examples
15+
4. Prioritise by impact (token cost > reliability > discoverability)
16+
17+
## The 10 AXI principles to check against
18+
19+
### Efficiency
20+
21+
- **P1 — Token-efficient output**: Use TOON format instead of JSON for agent-readable output. Omit braces, quotes, and commas. Example: `events[3]{tool_name,event_type,created_at}:` followed by plain rows.
22+
- **P2 — Minimal default schemas**: Return 3–4 fields by default; offer `--fields` or `--full` to expand.
23+
- **P3 — Content truncation**: Truncate long strings with a size hint like `(truncated, 847 chars — use --full to see more)`.
24+
25+
### Robustness
26+
27+
- **P4 — Pre-computed aggregates**: Always include `totalCount`, success rates, abandonment rates. Never make the agent round-trip for summaries.
28+
- **P5 — Definitive empty states**: Always print an explicit zero-result message. Never return empty output — agents cannot distinguish empty from failure.
29+
- **P6 — Structured errors & exit codes**: Mutations must be idempotent. Errors to stdout (structured), debug to stderr. Exit 0 on success, 1 on error. Never prompt interactively.
30+
31+
### Discoverability
32+
33+
- **P7 — Ambient context**: Self-install into session hooks. Output a compact dashboard so agents see current state before acting.
34+
- **P8 — Content first**: Running with no args should show live data, not help text. Include the current executable path and a one-sentence description.
35+
- **P9 — Contextual disclosure**: After every output, append `hint[]` lines with concrete next-step command templates. Carry forward fixed flags; leave runtime values as `<placeholder>`.
36+
- **P10 — Consistent `--help`**: Every subcommand must have a concise `--help` flag as fallback.
37+
38+
## Project context
39+
40+
This project's tooling scripts live in `scripts/tooling/`:
41+
42+
- `db.ts` — SQLite open/init (write only, AXI P6 applies to error handling)
43+
- `events.ts``trackEvent()` write function (P6: structured errors, idempotent)
44+
- `tool-usage-collection.ts` — CLI write wrapper called by hooks (P6: exit codes, P9: optional hint on success)
45+
- `tooling-mcp-server.ts` (Phase 2) — MCP write endpoint (P6: structured response with event_id)
46+
- `report.ts` (Phase 3) — Human + agent-facing inspect/report CLI (P1, P4, P5, P8, P9 all apply)
47+
- `check-skill-reporting-compliance.sh` (Phase 4) — Compliance checker (P5: always print count, P6: exit codes)
48+
49+
## Output format for your reviews
50+
51+
For each file, produce:
52+
53+
```
54+
## <filename>
55+
56+
violations[N]:
57+
P<n> — <principle name>: <what is missing>
58+
fix: <concrete code or command change>
59+
60+
suggestions[M]:
61+
P<n> — <principle name>: <nice-to-have improvement>
62+
example: <code snippet>
63+
```
64+
65+
If a file is fully compliant, output: `compliant: true — no AXI violations found`.
66+
67+
## Key trade-offs for this project
68+
69+
- `tool-usage-collection.ts` is intentionally **silent by default** (called from yarn hooks that must not pollute stdout). Suggest P9 hints only behind an opt-in `--verbose` flag.
70+
- `report.ts` is primarily **human-facing** (@clack/prompts UI), but should support `--agent` flag for TOON output (future MVP agent query interface).
71+
- The MCP `track_event` tool **must return a structured response** — even a minimal `{event_id, created_at}` confirmation — so the calling agent can verify the write succeeded.

package.json

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -562,6 +562,7 @@
562562
"@testing-library/react": "14.0.0",
563563
"@testing-library/react-hooks": "^8.0.1",
564564
"@testing-library/react-native": "^13.2.0",
565+
"@types/better-sqlite3": "^7.6.13",
565566
"@types/bnjs4": "npm:@types/bn.js@^4.11.6",
566567
"@types/bnjs5": "npm:@types/bn.js@^5.2.0",
567568
"@types/enzyme": "^3.10.12",
@@ -609,6 +610,7 @@
609610
"babel-plugin-transform-inline-environment-variables": "^0.4.4",
610611
"babel-plugin-transform-remove-console": "6.9.4",
611612
"base64-js": "^1.5.1",
613+
"better-sqlite3": "^12.8.0",
612614
"chalk": "^4.1.2",
613615
"chromedriver": "^123.0.1",
614616
"depcheck": "^1.4.7",
@@ -670,6 +672,7 @@
670672
"simple-git": "^3.25.0",
671673
"tailwindcss": "^3.4.0",
672674
"ts-node": "^10.9.2",
675+
"tsx": "^4.21.0",
673676
"typescript": "~5.4.5",
674677
"webdriverio": "^9.27.0",
675678
"webextension-polyfill": "^0.12.0",
@@ -781,6 +784,11 @@
781784
"eslint-import-resolver-typescript>unrs-resolver": false
782785
}
783786
},
787+
"dependenciesMeta": {
788+
"better-sqlite3": {
789+
"built": true
790+
}
791+
},
784792
"packageManager": "yarn@4.10.3",
785793
"foundryup": {
786794
"binaries": [

scripts/tooling/db.test.ts

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
import fs from 'fs';
2+
import os from 'os';
3+
import path from 'path';
4+
import { openDb, DEFAULT_DB_PATH, DEFAULT_DB_DIR } from './db';
5+
6+
describe('DEFAULT_DB_PATH', () => {
7+
it('points inside the home directory', () => {
8+
expect(DEFAULT_DB_PATH).toContain(os.homedir());
9+
expect(DEFAULT_DB_PATH).toContain('events.db');
10+
expect(DEFAULT_DB_DIR).toContain('.tool-usage-collection');
11+
});
12+
});
13+
14+
describe('openDb', () => {
15+
it('creates the events table with all required columns', () => {
16+
const db = openDb(':memory:');
17+
18+
const columns = db.prepare('PRAGMA table_info(events)').all() as {
19+
name: string;
20+
}[];
21+
const names = columns.map((c) => c.name);
22+
23+
expect(names).toEqual([
24+
'event_id',
25+
'tool_name',
26+
'tool_type',
27+
'event_type',
28+
'repo',
29+
'agent_vendor',
30+
'success',
31+
'duration_ms',
32+
'metadata',
33+
'created_at',
34+
]);
35+
36+
db.close();
37+
});
38+
39+
it('is idempotent — calling openDb twice on the same DB does not throw', () => {
40+
const tmpPath = path.join(
41+
os.tmpdir(),
42+
`tool-usage-idempotent-${Date.now()}.db`,
43+
);
44+
const db1 = openDb(tmpPath);
45+
db1.close();
46+
47+
// A second open should apply CREATE TABLE IF NOT EXISTS without error
48+
const db2 = openDb(tmpPath);
49+
db2.close();
50+
51+
fs.unlinkSync(tmpPath);
52+
});
53+
54+
it('enforces the event_type CHECK constraint', () => {
55+
const db = openDb(':memory:');
56+
57+
expect(() => {
58+
db
59+
.prepare(
60+
`INSERT INTO events
61+
(event_id, tool_name, tool_type, event_type, created_at)
62+
VALUES (?, ?, ?, ?, ?)`,
63+
)
64+
.run('id-1', 'my-tool', 'skill', 'invalid', new Date().toISOString());
65+
}).toThrow();
66+
67+
db.close();
68+
});
69+
70+
it('accepts valid event rows', () => {
71+
const db = openDb(':memory:');
72+
73+
const event = {
74+
event_id: 'abc-123',
75+
tool_name: 'worktree-create',
76+
tool_type: 'skill',
77+
event_type: 'start',
78+
repo: 'MetaMask/metamask-mobile',
79+
agent_vendor: 'cursor',
80+
success: null,
81+
duration_ms: null,
82+
metadata: null,
83+
created_at: '2026-04-09T10:00:00.000Z',
84+
};
85+
86+
db.prepare(`
87+
INSERT INTO events
88+
(event_id, tool_name, tool_type, event_type, repo, agent_vendor, success, duration_ms, metadata, created_at)
89+
VALUES
90+
(@event_id, @tool_name, @tool_type, @event_type, @repo, @agent_vendor, @success, @duration_ms, @metadata, @created_at)
91+
`).run(event);
92+
93+
const row = db
94+
.prepare('SELECT * FROM events WHERE event_id = ?')
95+
.get('abc-123') as typeof event;
96+
97+
expect(row).toMatchObject(event);
98+
99+
db.close();
100+
});
101+
102+
it('enforces PRIMARY KEY uniqueness', () => {
103+
const db = openDb(':memory:');
104+
105+
const insert = db.prepare(
106+
`INSERT INTO events (event_id, tool_name, tool_type, event_type, created_at) VALUES (?, ?, ?, ?, ?)`,
107+
);
108+
insert.run('dup-id', 'tool', 'skill', 'start', new Date().toISOString());
109+
110+
expect(() => {
111+
insert.run('dup-id', 'tool', 'skill', 'start', new Date().toISOString());
112+
}).toThrow();
113+
114+
db.close();
115+
});
116+
117+
it('creates the db directory when it does not exist', () => {
118+
const tmpDir = path.join(
119+
os.tmpdir(),
120+
`tool-usage-newdir-${Date.now()}`,
121+
'nested',
122+
);
123+
const tmpDb = path.join(tmpDir, 'events.db');
124+
125+
expect(fs.existsSync(tmpDir)).toBe(false);
126+
127+
const db = openDb(tmpDb);
128+
expect(fs.existsSync(tmpDb)).toBe(true);
129+
db.close();
130+
131+
fs.rmSync(path.dirname(tmpDir), { recursive: true });
132+
});
133+
});

scripts/tooling/db.ts

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
import Database from 'better-sqlite3';
2+
import fs from 'fs';
3+
import os from 'os';
4+
import path from 'path';
5+
6+
export const DEFAULT_DB_DIR = path.join(os.homedir(), '.tool-usage-collection');
7+
export const DEFAULT_DB_PATH = path.join(DEFAULT_DB_DIR, 'events.db');
8+
9+
/**
10+
* Opens (or creates) the tool-usage SQLite database, initialises the schema,
11+
* and enables WAL mode for safe concurrent writes from multiple processes.
12+
*
13+
* Pass ':memory:' during tests to avoid touching the filesystem.
14+
*/
15+
export function openDb(dbPath = DEFAULT_DB_PATH): Database.Database {
16+
// WAL journal mode requires a real file; skip dir creation and pragma for memory DBs
17+
const isMemory = dbPath === ':memory:';
18+
19+
if (!isMemory) {
20+
const dir = path.dirname(dbPath);
21+
if (!fs.existsSync(dir)) {
22+
fs.mkdirSync(dir, { recursive: true });
23+
}
24+
}
25+
26+
const db = new Database(dbPath);
27+
28+
if (!isMemory) {
29+
db.pragma('journal_mode = WAL');
30+
}
31+
32+
db.exec(`
33+
CREATE TABLE IF NOT EXISTS events (
34+
event_id TEXT PRIMARY KEY,
35+
tool_name TEXT NOT NULL,
36+
tool_type TEXT NOT NULL,
37+
event_type TEXT NOT NULL CHECK(event_type IN ('start', 'end')),
38+
repo TEXT,
39+
agent_vendor TEXT,
40+
success INTEGER,
41+
duration_ms INTEGER,
42+
metadata TEXT,
43+
created_at TEXT NOT NULL
44+
)
45+
`);
46+
47+
return db;
48+
}

0 commit comments

Comments
 (0)