Rebuild agent skills around what the repo can't say about itself; measure every cut (closes #525)

[fangge518]

Why most of the old skill text is gone

The agents these files serve can recover most facts about the codebase by reading it — faster than we can keep prose about it true. Text that restates the code is a drift-prone paraphrase that pays context rent on every load. So this PR reorganizes the agent-facing corpus around one question: what can the repository not say about itself? Three kinds of content survive it:

• Procedures — command sequences with verify steps (the runbooks).
• Norms and values — the forbidden-but-possible, stated at the most fundamental level that still permits inference (a security boundary, a deploy-authority rule), including standards the code sometimes violates.
• Terminology seams — where a name misleads (map_type: "default" means a district map) or one concept has several names (shatter/"Break"/"Super Draw").

Everything else moved to a better home or was cut. A norm-by-norm review of the previous eight knowledge skills found most written "invariants" were observations promoted to norms: descriptions of the file layout ("all subscriptions wire through one file" — three standing counterexamples, one of them correct design), architecture facts whose violation would require tooling absent from the backend. Of ~20 candidate norms, six survived as genuinely normative and non-derivable. The rest were cut outright, moved to orientation docs, or pushed to the one place a future editor must look anyway (a comment at the choke point — the ParentChildEdges declaration, the sync-merge override).

The resulting shape:

• 5 norm skills (backend-endpoints, map-rendering, map-edit-sync, performance-memory, deploy-authority), each 20–50 lines: norms plus the vocabulary bound to their situations. deploy-authority and run-dependency-audit are paths-gated (auto-load on infra/**/workflow and manifest edits).
• 5 runbooks, renamed with a run- prefix so procedure triggers read differently from norm triggers. Two of the original seven were cut: run-next-upgrade (pure official-docs paraphrase for a rare task an agent can rederive from the live guide) and run-pr-review (a compiled list of other skills' triggers — the router already serves each checkpoint). run-quality-gate later dropped its runner script entirely: gate-call traces from live sessions showed agents running every gate indiscriminately, so the skill is now judgment-first — decide per gate from the situation (a two-case rule for pytest, since CI reruns it on every backend push), then run standalone commands.
• docs/overview.md — newcomer orientation: what the system is, the full name-concordance table (12 rows), the repository layout, how modules/plans/rendering/sync/auth/CMS fit — and docs/decisions.md — the dated history of architectural decisions (departitioning, the graph rework, undo-per-gesture, computation placement) with their grounds. .agents/ARCHITECTURE.md also moved to docs/architecture.md (its stale Fly.io section fixed), and the skills source tree was renamed .agents/skills/ → agent-skills/ — no harness reads .agents/, so the dir is now visible and self-describing. Orientation lacks a moment of need to route on, so it lives where humans and agents read eagerly, outside the skill layer. These docs are seeds: harvested from the cut skill material and verified, but building them into full documentation is deliberately out of scope here — a future PR's work.
• CLAUDE.md gains exactly two cross-cutting rules — the document-UUID-as-secret norm and the map_type: "default" trap — the only two items whose need is undetectable (an agent confidently wrong gets no retrieval signal).
• agent-skills/AUTHORING.md now states the doctrine the above applies: the content test, placement by detectability of need, one skill per trigger, pointer/reference principles (a pointer's condition must be checkable at the moment it's read), and a revision procedure that grows the floor only on trace evidence.

Evidence the cuts don't hurt: adversarial evals on the strongest cut content

Cuts were then challenged, not assumed. Four eval cases (official claude plugin eval format; run by hand pending early access) each target cut content with the strongest prior claim to being load-bearing, phrased adversarially — a task that invites exactly the mistake the cut line used to guard against. Each ran in fresh sessions against two isolated repo copies: the rebuilt skills vs. the pre-rebuild skills (2 trials per arm, plus a no-skills smoke run; LLM-rubric graded). Suite, rubrics, and all 17 transcripts are preserved at https://github.com/districtr/districtr-scratchpad/tree/main/skill-evals/2026-09-agent-skills-rebuild.

Case (cut under test)	Rebuilt	Old	Outcome
Repartition the assignments tables? (departitioning history → docs)	2/2	2/2	Safe — every run found PR #625's shipped-measured-reverted history (via docs/decisions.md, the migration's own docstring, or the old skill) and recommended against.
Debug a silent county-brush failure on a new module (onboarding-metadata warnings cut)	2/2	2/2	Safe — all four ranked the unfilled statefps column (open #633) in their top two causes from code alone.
Reuse the hover feature-state key from a sidebar? (per-key ownership rule cut as generic MapLibre lore)	2/2	2/2	Safe, decisively — every run identified the two-writer clobber unprompted and designed around it better than the cut rule specified.
"Simplify" the sync merge that pins overlays/statefps to the server (server-owned-fields rule → docs)	1/2	0/2	The lesson. Pass/fail tracked skill routing, not content: with map-edit-sync loaded, agents refused and cited the docs; with no skill loaded (including both old-skill trials — the old inventory never routed on this framing either), agents confidently deleted the guard.

Three of four cuts confirmed safe. The fourth showed the guarded knowledge survives only when routing succeeds — so the durable fix went where routing can't miss: a why-comment at the merge site in fetchDocument.ts, which both passing runs independently proposed. The suite also caught real repo issues along the way: the statefps onboarding gap is now a step-level runbook warning, and a genuine event_id/eventId contract mismatch surfaced (left for a follow-up issue; the audit-driven fix was drafted and deliberately withdrawn from this PR).

The eager/lazy split of the always-loaded files

The same doctrine was then applied to the memory files themselves. Root AGENTS.md (the cross-tool standard name, as in PR #719) now carries the agent instructions, and CLAUDE.md is a three-line @AGENTS.md import — one source, so the two can't drift. Because that file is loaded eagerly into every session, it was pruned to what the official CLAUDE.md guidance admits: three cross-cutting rules (document UUID, map_type: "default", Bun-not-npm), the quality-gate commands, and the pointers that make the lazy layers discoverable. Everything the bd prime SessionStart hook already injects (command reference, session-close protocol) was deduplicated away, along with stack descriptions readable from the lockfiles and the gate commands (now a pointer to run-quality-gate, their one home). The lazily-read hub (.agents/AGENTS.md) went through the same pass — three inconsistent bd sections collapsed, the generated troubleshooting catalog cut, "Frontend Conventions" dissolved (one line promoted to the eager file: Bun-not-npm), the directory tree moved to docs/overview.md, the skills listing cut (agents receive every description at session start; the list was a drifting paraphrase) — at which point its only unique facts were "base work on dev" (now a cross-cutting rule) and the skills sync workflow (now in AUTHORING's mechanics), so the hub file was deleted. The always-loaded surface is a single ~30-line root AGENTS.md: four pointers, three cross-cutting rules (document UUID, Bun, base-on-dev), a gates pointer, and a minimal beads note.

The delivery bug that started all this

scripts/sync-skills.sh mirrored the source tree into .claude/skills/project/<name>/, but Claude Code discovers skills exactly one level deep — every learn-* skill had been outside native routing since the April scaffold. Fix: the Claude sync target flattens to .claude/skills/<name>/ (with a duplicate-name guard), and the Cursor target's reference-inlining matches references/ as the files actually name it. The source tree is flat too. Verified live: all current skills appear in a fresh session's listing after sync.

Appendix: the earlier measurements

Before the norm review, two experiment sets (2026-09-01, isolated claude -p harness, ~220 runs total) grounded AUTHORING.md's routing and content doctrine:

Routing (~170 runs, Haiku + Sonnet): the model matches skills against its own one-sentence restatement of the task. Grammatical phrases in plain engineering nouns triggered 4/4 under every name/description split; the same words scrambled, 0/12; house terms never. A wrong description vetoes invocation (0/11). Two-concern tasks loaded both intended skills 12/12 across three pairs. Haiku consulted the listing essentially never (3/72 direct, 0/20 indirect — even with the skill's territory files open), and subagents invoked a skill once across the whole experiment: worker spawns carry their hard invariants in the prompt.

Point ablations (two rounds, 20 randomly sampled points, skill force-loaded with and without the single point; round-two verdicts predicted in advance, 8/10): 3 load-bearing, 12 derivable, 4 inert, 1 marginal. Every load-bearing point was arbitrary team policy; every generic-lore point tested derivable. One caution became doctrine: the study's strongest point (3/3 vs 0/3 behavior flip) steered toward a rule later judged wrong and rewritten — ablation measures leverage; review measures truth.

Tested

• Flat sync verified end to end: current inventory (5 norm skills + 5 runbooks) at one level in .claude/skills/, hot-loaded into a live session's listing
• Challenge evals: 17 transcripts across 4 adversarial cases, two skill inventories (results above; suite in official plugin-eval format, runnable natively once early access lands)
• Gate commands exercised for real (the runner script ran against a real diff before being deleted in favor of standalone commands); audit_contracts.py executed against the real repo (86 backend models, 26 frontend types — and it found a real contract mismatch)
• bun outdated / audit commands timed (~1.4s); uvx pip-audit host crash found and container fallback documented in the runbook
• docker-compose up pre-commit green on every commit (hook-enforced)
• Cursor sync inlines references/*.md; Codex index lists all skills

[nofurtherinformation]

Overall looks good and a really solid refactor + update. Just a few minor things that should be cleaned up, but marking as approved to avoid blocking unnecessarily.

It looks like these docs were made before the graph changes landed fully, so there are a few stale references to shatter_parent.sql. A quick pass to remove extra markdown line breaks would be preferred, too.

[nofurtherinformation]

H: shatter_parent.sql is no longer active, replaced by api/gerrydb/edges/{districtr map slug}?parent_geoid={vtd:123}

[fangge518]

Confirmed and fixed — you're right, and the same merge that brought this in also dropped the UDFs (migration 2ecf1bdc582b), which I'd missed. Rewrote the contract clause to name the live paths: interactive shatter is client-side from graph children served by GET /api/gerrydb/edges/{slug}, CSV import heals/fills via _heal_or_fill. Also corrected the same stale claim in docs/decisions.md and run-map-onboarding (parentchildedges is now write-only pending its drop).

[nofurtherinformation]

H: Noted elsewhere, too -- I think shatter_parent is not used in live paths now?

[nofurtherinformation]

PP: A quick pass to remove hard line breaks from the MD could be good, but non-critical.

[fangge518]

Done — unwrapped hard line breaks across the branch's markdown (skills, docs, AGENTS.md); paragraphs are single lines now.

[nofurtherinformation]

I: This seems correct. But, at what point is it worth considering something like HeyAPI?

[fangge518]

Agreed — generating types from the OpenAPI schema is the real fix, and this audit script is explicitly a stopgap for the hand-maintained types.ts. I filed #756 for HeyAPI; the skill now notes it retires if that lands.

[nofurtherinformation]

H: I think this might mask the error exit code, we might want:

Suggested change

	if docker-compose up pre-commit >"${pre_commit_log}" 2>&1; then
	if docker-compose run --rm pre-commit >"${pre_commit_log}" 2>&1; then

[fangge518]

Applied — run-gates.sh has since been deleted entirely (the skill became judgment-first knowledge with standalone commands), but the docker-compose up pre-commit line had migrated into the skill's Commands block; it's now docker-compose run --rm pre-commit.