fix: time out hung gh exec in capacity wait

[SebTardif]

scripts/lib.mjs ghRaw called execFileSync(gh, ..., --paginate --slurp) with no timeout. If that GitHub CLI child never returns, waitForLiveWorkerCapacity cannot reach its 30 minute deadline and dispatch stays pinned until someone kills it.

What happens

1. Throttled dispatch and other callers poll live worker runs through waitForLiveWorkerCapacity.
2. Each poll uses gh api .../runs --paginate --slurp via ghRaw / execFileSync.
3. Those children had no timeout, so a stalled gh never returns.
4. The 30 minute capacity deadline is checked only between polls, so it never fires.

This is separate from dispatch secret/variable list timeouts (#323) and dispatch child runCommand timeouts (#324).

Proposed fix

Give ghRaw the same timeout contract as worker planner/review children:

• CLOWNFISH_GH_EXEC_TIMEOUT_MS (default 2 minutes)
• execFileSync(..., { timeout, killSignal: "SIGKILL" }) for capacity gh polls
• Cap each poll by the remaining waitForLiveWorkerCapacity deadline
• On ETIMEDOUT, keep polling until that deadline, then throw the existing capacity timeout

A source contract plus hung-gh and timely-list fixtures live in test/lib.test.mjs.

Fork branch

This repo limits pull requests to collaborators, so I could not open a PR. The patch is on my fork:

• Branch: SebTardif:fix/capacity-gh-exec-timeout
• Compare: main...SebTardif:fix/capacity-gh-exec-timeout
• Commit: eeece3d86

Feel free to cherry-pick that commit, or I can open a PR if you add this account as a collaborator.

[clawsweeper]

Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed September 2, 2026, 5:45 PM ET / 21:45 UTC.

Summary
Keep open and route to a focused repair: current main checks the capacity deadline only outside the synchronous GitHub CLI poll, while that poll has no timeout, so a hung child can prevent the deadline from being observed.

Reproducibility: yes. source-reproducible: a GitHub CLI executable that does not exit is invoked synchronously from the capacity loop without a timeout, preventing the loop from rechecking its deadline.

Next step
Current source proves a narrow, isolated capacity-wait defect with clear implementation and regression-test boundaries; the related timeout issues cover different subprocess paths.

Review details

Best possible solution:

Thread the remaining capacity deadline into each capacity-query subprocess, terminate only an overlong GitHub CLI call, and add a regression fixture proving the existing capacity-timeout error is raised promptly.

Do we have a high-confidence way to reproduce the issue?

Yes, source-reproducible: a GitHub CLI executable that does not exit is invoked synchronously from the capacity loop without a timeout, preventing the loop from rechecking its deadline.

Is this the best way to solve the issue?

Yes: bounding the existing child invocation by the remaining capacity deadline is narrower and safer than changing dispatch semantics or adding a separate polling path.

AGENTS.md: found and applied where relevant.

Remaining risk / open question:

• The repair must retry only ETIMEDOUT during a capacity wait; authentication, API, and malformed-response failures should retain their current fail-fast behavior.

Codex review notes: model internal, reasoning high; reviewed against 19a2f621a93c.

Label changes

Label changes:

• add P1: A hung GitHub CLI call can block normal throttled dispatch and recovery automation until an operator intervenes.
• add impact:crash-loop: The affected capacity-wait path can hang the dispatch process rather than reaching its configured timeout.
• add issue-rating: 🦞 diamond lobster: Current issue advisory state selects this label.
• add clawsweeper:source-repro: Current issue advisory state selects this label.
• add clawsweeper:queueable-fix: Current issue advisory state selects this label.
• add clawsweeper:fix-shape-clear: Current issue advisory state selects this label.

Label justifications:

• P1: A hung GitHub CLI call can block normal throttled dispatch and recovery automation until an operator intervenes.
• impact:crash-loop: The affected capacity-wait path can hang the dispatch process rather than reaching its configured timeout.

Evidence reviewed

Acceptance criteria:

• node --test test/lib.test.mjs test/dispatch-jobs-capacity.test.mjs
• npm run validate

What I checked:

• Unbounded capacity poll: waitForLiveWorkerCapacity establishes a deadline but calls liveWorkerCapacity synchronously inside the loop; the deadline is only evaluated before and after that call. (scripts/lib.mjs:78, 19a2f621a93c)
• GitHub CLI has no timeout: The sole ghRaw call path used by listActiveWorkflowRuns invokes execFileSync without timeout or killSignal, so a stalled GitHub CLI cannot return control to the capacity loop. (scripts/lib.mjs:162, 19a2f621a93c)
• Affected dispatch path: The dispatcher calls waitForLiveWorkerCapacity when --wait-for-capacity is selected, making this reachable from normal throttled dispatch. (scripts/dispatch-jobs.mjs:243, 19a2f621a93c)
• Existing capacity coverage omits a hung child: The capacity tests exercise delayed GitHub visibility with a responding fake CLI, but contain no hung-CLI timeout scenario. (test/dispatch-jobs-capacity.test.mjs:9, 19a2f621a93c)
• Related timeout convention: The worker already bounds planner subprocesses with timeout and SIGKILL, providing a local precedent for child-process containment. (scripts/run-worker.mjs:82, 19a2f621a93c)
• Feature-history routing: History for scripts/lib.mjs includes Vincent Koc's capacity-cap and pagination work, including commit bde7630. (scripts/lib.mjs:42, bde7630054b9)

Likely related people:

• Vincent Koc: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

How this review workflow works

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.