feat(skills): driver state machine, prompt-on-failure capture, activity-feed watcher (#3)

endolinbot · claude · endolinbot · commit 523896849a0b · 2026-06-02T23:14:25.000Z
Rename skills/driver-state-machine to skills/driver-pr-creation-state-machine
to fit the per-workflow naming convention (every workflow lives at
skills/driver-&lt;kind&gt;-state-machine/SKILL.md). The body is preserved
verbatim apart from the renamed title and a redirect note explaining
the move. Updated the two callers that referenced the prior name
(skills/cleaner/SKILL.md, skills/driver-design-only-pr-workflow/SKILL.md).

Two new skill files:

  skills/prompt-on-failure-capture/SKILL.md     the git hash-object /
                                                cat-file blob capture
                                                pattern, prompt template,
                                                claude invocation, and
                                                known-SHA short-circuit
                                                (per designs/driver.md §
                                                Prompt-on-failure capture
                                                pattern).

  skills/activity-feed-watcher/SKILL.md         the contract every
                                                per-feed watcher daemon
                                                implements: event
                                                classification,
                                                subscription-union
                                                routing, deterministic
                                                reactji posting, error
                                                escalation (per
                                                designs/driver.md §
                                                Watcher subscription
                                                model and event routing).

Phase 1 scope of the migration plan. The other named skills
(gardener-inbox-error-reporting, driver-pre-ci-validation, and per-
workflow state machines for observed-error / issue-response /
build-request / design-request / retcon-rebase / ci-recovery) are
deferred to Phase 2-5.

Co-Authored-By: Claude Opus 4.7 (1M context) &lt;noreply@anthropic.com&gt;
diff --git a/skills/activity-feed-watcher/SKILL.md b/skills/activity-feed-watcher/SKILL.md
@@ -0,0 +1,168 @@
+---
+created: 2026-06-02
+updated: 2026-06-02
+author: builder
+---
+
+# Skill: activity-feed-watcher
+
+The contract every per-feed activity watcher implements: event
+classification, subscription-union routing, deterministic reactji
+posting, and error escalation. Each
+[`scripts/watcher/<feed>/watcher.sh`](../../scripts/watcher/README.md)
+consults this skill on the LLM-substep paths it has (most paths are
+deterministic and skill-free).
+
+See [`designs/driver.md`](../../designs/driver.md) § Watcher
+subscription model and event routing for the design rationale; this
+skill is the playbook the implementation follows.
+
+## When to use
+
+When implementing a new feed watcher (a per-repo webhook stream, a
+poll-fallback loop, the review-queue endpoint, an assigned-issues
+feed). The skill is the contract; the watcher script is the
+implementation.
+
+## Inputs
+
+- A feed source the watcher polls or subscribes to (a webhook
+  endpoint, a `gh api` poll loop, a kriskowal-side endpoint, etc.).
+- Read access to the union of all driver subscription files at
+  `journal/drivers/<host>/<lane>.subscriptions`.
+- Write access to the journal for:
+  - per-PR event logs at `journal/events/<repo>--<pr>.log`;
+  - jobs posted to `journal/jobs/open/` for events with no
+    subscribed driver;
+  - gardener-inbox messages for unrecoverable feed failures.
+
+## State
+
+- **Last-seen-id cache** (per-feed, outside the dispatch root):
+  `~/.garden-watchers/<feed>/last-seen-id` (or analogous
+  per-feed-name path). The watcher reads it on every tick to know
+  what events are new; updates it after the events for one tick
+  are routed.
+- **ETag cache** (for `If-None-Match`-honoring HTTP feeds): same
+  shape as the existing standing-monitor daemons'
+  `/tmp/garden-monitor-*` caches; one ETag per polling endpoint.
+- **Subscription union**: read fresh from
+  `journal/drivers/<host>/<lane>.subscriptions` on every tick; the
+  watcher does not cache the union because subscriptions change
+  with driver lifecycle.
+
+## Procedure
+
+A watcher's loop per tick:
+
+1. **Refresh subscriptions.** Read every
+   `journal/drivers/<host>/<lane>.subscriptions` file. Build the
+   union: a map from `<repo>:<pr>` to the set of subscribing
+   lane numbers. The watcher uses this map for the routing
+   decision in step 4.
+
+2. **Poll the feed.** Fetch the new events since
+   `last-seen-id` (or `If-None-Match` against the cached ETag).
+   On a transient error (network failure, 5xx), log and skip the
+   tick; systemd's `Restart=on-failure` and the next tick's poll
+   resolve the case.
+
+3. **Classify each event.** Each watcher implements a deterministic
+   classifier keyed on the event-type field of its feed:
+
+   | Event kind                       | Classification             |
+   | -------------------------------- | -------------------------- |
+   | New comment on a PR              | `comment`                  |
+   | Review submission                | `review`                   |
+   | Push to a PR's head ref          | `push`                     |
+   | CI status change                 | `ci-status`                |
+   | Label change                     | `label`                    |
+   | Assigned-issue update            | `assigned-issue`           |
+   | Issue body mention of the bot    | `issue-mention`            |
+   | Other                            | `other` (logged, not routed) |
+
+   Classification is regex / structural; no LLM is involved. The
+   classifier's escalation surface is empty by design.
+
+4. **Route the event.** For each classified event, decide its
+   destination:
+
+   - If the event names a `<repo>:<pr>` that appears in the
+     subscription union, append a message to
+     `journal/events/<repo>--<pr>.log` for every subscribed lane
+     (a single append is enough; per-lane fan-out happens via the
+     driver's read).
+   - If the event names a `<repo>:<pr>` *not* in the subscription
+     union, post a job to `journal/jobs/open/` with `kind:
+     pr-creation` (so any eligible driver lane can claim and bind).
+   - If the event is an `assigned-issue` event, post a job with
+     `kind: issue-response`.
+   - If the event is `other`, log to the watcher's transcript and
+     drop.
+
+5. **Post the deterministic reactji.** For every `comment` event
+   on any covered repo (subscribed PR or not), the watcher posts
+   the `:eyes:` reactji synchronously, *before* the route in step
+   4 lands. The `(comment_id, reaction_id)` pair is recorded in
+   the destination event log so downstream agents do not
+   double-post.
+
+6. **Persist last-seen.** Update the per-feed `last-seen-id` cache
+   so the next tick starts from this point.
+
+7. **Sleep until the next tick.** The polling cadence is per-feed
+   (a webhook stream may have no sleep; a `gh api` poll has a 30-
+   to 60-second tick budget).
+
+## Output
+
+- Per-PR event log entries at `journal/events/<repo>--<pr>.log`
+  (one append per routed event).
+- Posted jobs at `journal/jobs/open/<...>.md` for unsubscribed
+  events.
+- Reactji on PR comments (the `:eyes:` deterministic acknowledgment).
+- Updated per-feed `last-seen-id` / ETag caches outside the journal.
+
+## Error escalation
+
+The watcher inherits the uniform error-reporting pattern from
+[`skills/gardener-inbox-error-reporting/SKILL.md`](../gardener-inbox-error-reporting/SKILL.md):
+
+- A `-x` subshell captures the per-tick transcript.
+- An `EXIT` / `ERR` trap discriminates on `$?`.
+- On unexpected exit, the trap hashes the transcript via
+  `git hash-object -w --stdin`, appends a section to
+  `journal/inboxes/<host>/gardener.md` naming the feed slug, the
+  transcript SHA, and a one-paragraph context, and exits non-zero.
+- systemd's `Restart=on-failure` (30-second backoff) brings the
+  watcher back automatically.
+
+Persistent crash loops accumulate sections in the gardener inbox;
+the maintainer's next pass sees the pattern and triages.
+
+## Notes
+
+- **One watcher per feed, not per repo.** Multiple repos behind one
+  webhook stream share a single watcher process; a repo with its
+  own poll cadence runs its own watcher.
+- **No LLM in the routing path.** The watcher is fully
+  deterministic. Classification, subscription union, reactji
+  posting, and per-PR event-log appends are all bash / regex.
+  Routing decisions are not gated on judgment.
+- **Subscription-union freshness.** A driver lane that just took
+  on a new subscription must write its `.subscriptions` file
+  *before* the watcher's next tick. The watcher does not lock or
+  serialize; the race is "the driver missed this tick's event,
+  the next tick will pick it up."
+- **Monitoring safety constraint.** Adding a new feed for a repo
+  whose comments and PRs are not gated against untrusted
+  contributors is gated on explicit maintainer authorization
+  recorded in a journal `message` entry, per
+  [the constraint in CLAUDE.md](../../CLAUDE.md#monitoring-safety-constraint).
+  Watcher daemons are an event-level surveillance surface and
+  inherit the same rule.
+- **Coalescence with existing standing monitors.** Phase 1 ships
+  one stub feed (`endo-but-for-bots`). Phase 4-5 of the migration
+  plan in `designs/driver.md` retires the per-repo standing
+  monitors (`/tmp/garden-monitor-*.log` poll daemons) on a per-
+  feed basis after observed equivalence.
diff --git a/skills/cleaner/SKILL.md b/skills/cleaner/SKILL.md
@@ -15,7 +15,7 @@ Phase 3 lands the real coverage-driven-testing body inside this script; phase 1
 
 ## When to use
 
-- The driver posts a `clean` job (per `skills/driver-state-machine/SKILL.md` § `clean` state).
+- The driver posts a `clean` job (per `skills/driver-pr-creation-state-machine/SKILL.md` § `clean` state).
 - A worker daemon polling either the flat board (`jobs/open/` with `eligible_roles:` containing `cleaner`) or the per-role board (`jobs/cleaner/open/`) claims the job and invokes this script with the claimed path.
 - The script does the work (phase 1: no-op stub; phase 3: coverage-driven-testing pass) and moves the job to `done/` or `abandoned/`.
 
diff --git a/skills/driver-design-only-pr-workflow/SKILL.md b/skills/driver-design-only-pr-workflow/SKILL.md
@@ -11,7 +11,7 @@ A design-only PR touches only paths under `<project>/designs/` (typically
 `designs/<slug>.md` plus minor README updates); it has no source-touching
 CI to wait on, no cleaner pass, and no fixer-loop. The state machine is
 the strict subset of
-[`skills/driver-state-machine/SKILL.md`](../driver-state-machine/SKILL.md)
+[`skills/driver-pr-creation-state-machine/SKILL.md`](../driver-pr-creation-state-machine/SKILL.md)
 that the design-only branch needs.
 
 This is the simplest workflow shape and the natural Phase-2 target per
diff --git a/skills/driver-pr-creation-state-machine/SKILL.md b/skills/driver-pr-creation-state-machine/SKILL.md
@@ -1,17 +1,23 @@
 ---
 created: 2026-06-01
-updated: 2026-06-01
+updated: 2026-06-02
 author: builder
 ---
 
-# Skill: driver-state-machine
+# Skill: driver-pr-creation-state-machine
 
 The PR-creation workflow's states and transition predicates. The driver
-loads this skill on entry to a PR-creation-bound subscription. Each
-state is a predicate the driver evaluates against live GitHub state plus
-its per-lane state file; each transition is either a deterministic
-action (run in-process) or a job-board post (delegated to a
-role-specific worker).
+loads this skill on entry to a PR-creation-bound subscription (or when
+it claims a job whose `kind:` is `pr-creation`). Each state is a
+predicate the driver evaluates against live GitHub state plus its
+per-lane state file; each transition is either a deterministic action
+(run in-process) or a job-board post (delegated to a role-specific
+worker).
+
+This file was renamed from `skills/driver-state-machine/SKILL.md` on
+2026-06-02 to fit the per-workflow naming convention introduced by the
+[scripts/ layout pivot](../../designs/driver.md) (every workflow lives
+at `skills/driver-<kind>-state-machine/SKILL.md`).
 
 For the smaller design-only PR variant (Phase 2 of the migration), see
 the sibling skill at
diff --git a/skills/prompt-on-failure-capture/SKILL.md b/skills/prompt-on-failure-capture/SKILL.md
@@ -0,0 +1,176 @@
+---
+created: 2026-06-02
+updated: 2026-06-02
+author: builder
+---
+
+# Skill: prompt-on-failure-capture
+
+The capture-by-SHA pattern that lets a deterministic bash script
+(driver, watcher, helper) escalate an unresolvable step to an
+ephemeral `claude -p` subagent without inlining a large failure log
+into the prompt. The log is hashed into the journal's git object
+database; the prompt names the SHA; the subagent reads the blob on
+demand via `git cat-file blob`.
+
+See [`designs/driver.md`](../../designs/driver.md) § Prompt-on-failure
+capture pattern for the design rationale and the four-slot brief
+shape.
+
+## When to use
+
+- A driver tick reaches a state whose deterministic predicate cannot
+  resolve the next transition (the canonical example: a panel
+  verdict body that says "almost-approve with one footnote" and the
+  predicate cannot tell if that is `approve` or `must-fix-loop`).
+- A watcher hits a feed event whose classification the regex
+  filter cannot resolve.
+- A helper script (cleaner skeleton, fixer's retcon wrapper) fails
+  in a way the surrounding bash cannot interpret.
+
+When the deterministic predicate succeeds, this skill is not used:
+the script advances on its own. The skill is the escape hatch, not
+the default.
+
+## Inputs
+
+- A failure context (a log file, a JSON payload, a diff, a verdict
+  body) the script captured during its own run.
+- The four-slot brief metadata (PR identifier, design path, role,
+  state machine state).
+- A one-paragraph human-authored reason for the escalation (the
+  context line the prompt template fills).
+
+## State
+
+The capture writes a blob to `$GARDEN_JOURNAL`'s object database
+via `git hash-object -w --stdin`. The blob is unreferenced; `git gc`
+collects it after the journal's grace window (default 14 days)
+unless an operator or agent anchors it:
+
+```sh
+git -C "$GARDEN_JOURNAL" update-ref refs/captures/<role>/<pr>/<short-id> <sha>
+```
+
+Captures under `refs/captures/` survive indefinitely until explicitly
+pruned. Most one-off failures do not need this. Anchoring is
+appropriate when:
+
+- The failure shape is one we want to revisit (a recurring CI
+  flake we want to round-trip into a regression test).
+- The escalation produced a non-trivial classification we want to
+  reuse on identical SHAs (see *Known-SHA short-circuit* below).
+- An operator's triage flagged it.
+
+A small lookup table at
+`journal/drivers/<host>/<lane>.classifications` may also map
+capture SHA → classification verdict so identical bodies reuse the
+prior verdict without re-invoking the LLM.
+
+## Procedure
+
+1. **Capture.** Write the relevant output to the journal as a blob:
+
+   ```sh
+   LOG_SHA=$(some_command 2>&1 | git -C "$GARDEN_JOURNAL" hash-object -w --stdin)
+   ```
+
+   The blob's content is content-addressable: identical inputs
+   produce identical SHAs. This is the foundation of the known-SHA
+   short-circuit below.
+
+2. **Promote (optional).** When the failure is one we want to
+   survive `git gc`, anchor it:
+
+   ```sh
+   SHORT_ID=$(printf '%s' "$LOG_SHA" | head -c 7)
+   git -C "$GARDEN_JOURNAL" update-ref \
+     "refs/captures/<role>/<pr>/${SHORT_ID}" "$LOG_SHA"
+   ```
+
+3. **Known-SHA short-circuit.** Before constructing the prompt,
+   consult the per-lane classifications table:
+
+   ```sh
+   CLASSIFICATIONS="$GARDEN_JOURNAL/drivers/$GARDEN_HOST/$LANE.classifications"
+   if [ -f "$CLASSIFICATIONS" ]; then
+     KNOWN=$(grep "^$LOG_SHA " "$CLASSIFICATIONS" | head -1 | cut -d' ' -f2-)
+     if [ -n "$KNOWN" ]; then
+       # apply the known disposition and skip the LLM call
+       apply_classification "$KNOWN"
+       return 0
+     fi
+   fi
+   ```
+
+   The match is exact; near-misses (a CI log with one timestamp
+   different) hash to a different SHA and re-escalate.
+
+4. **Prompt construct.** Write a prompt file that fills the four
+   named slots plus the capture SHA and the failure context:
+
+   ```
+   You are a subagent classifying an escalation from driver lane $LANE.
+
+   PR:        $DRIVER_PR
+   Design:    $DESIGN_PATH                  # or "(none)"
+   Role:      $ESCALATING_ROLE              # builder, fixer, etc.
+   State:     $DRIVER_STATE
+   Capture:   $LOG_SHA
+
+   Context: $FAILURE_CONTEXT_PARAGRAPH
+
+   Read the capture on demand via:
+     git -C "$GARDEN_JOURNAL" cat-file blob $LOG_SHA
+
+   Reply with a structured JSON verdict naming the classification and,
+   when applicable, the action the driver should take next.
+   ```
+
+   The four-slot brief is the minimum. Some workflows add a fifth
+   slot (the design dependency walk's prior-PR list, the conflict
+   resolver's parent SHAs).
+
+5. **Invoke.** Pipe the prompt to `claude -p`:
+
+   ```sh
+   RESPONSE=$(printf '%s' "$PROMPT" | claude -p --output-format json)
+   ```
+
+   The prompt is small (the four-slot brief plus the context
+   paragraph); the LLM pulls the capture on demand. Token cost
+   stays bounded by the brief, not the capture's length.
+
+6. **Apply.** Read the response (a patch, a directive, or a
+   classification), apply it, and record the SHA → verdict mapping
+   in the per-lane classifications table so the next identical
+   capture short-circuits.
+
+## Output
+
+- The capture blob's SHA (written to the journal's object DB).
+- (Optional) An anchored ref under `refs/captures/<role>/<pr>/<short-id>`.
+- The LLM's classification or patch.
+- An updated `journal/drivers/<host>/<lane>.classifications` row when
+  the SHA was unseen.
+
+## Notes
+
+- **Identical SHAs short-circuit.** The whole point of the
+  capture-by-SHA pattern is that recurring operational flakes (a
+  test-xs esvu-download failure, an esbuild bundling error with a
+  stable error trail) hash to the same blob and reuse the prior
+  classification.
+- **The prompt stays small.** Inlining the log defeats the
+  capture's purpose and reintroduces the per-cycle token cost the
+  driver design eliminates. The prompt names the SHA and lets the
+  subagent decide if it needs to read the capture.
+- **Sensitive content.** Captures persist in the journal repo
+  until `git gc` collects them. Secrets that leaked into a
+  transcript stay readable until the grace window passes. Do not
+  anchor a capture (`refs/captures/...`) unless its content is
+  safe to retain indefinitely.
+- **No promotion to a tracked file.** Captures live as blobs only;
+  there is no `journal/captures/` tree of committed snapshots.
+  The `git hash-object -w` plus optional `update-ref` shape is
+  the contract.
