fix(worker): stop silently skipping pre-PR checks when dependencies are missing

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015kfeohXE66xx7RJPxZ2pvH

Author: outof-place

apps/worker/src/pre-pr-checks/runner.test.ts

@@ -177,6 +177,40 @@ describe("runPrePrChecksWithFixes", () => {
     expect(result.failures).toHaveLength(1);
   });
 
+  it("fails a check that exits 0 while reporting its dependencies are not installed", async () => {
+    mockRunCommand.mockImplementation((cmd, args) => {
+      if (cmd === "cat" && args[0] === WORKSPACE_MANIFEST_PATH) {
+        return commandResult(0, JSON.stringify(manifest));
+      }
+      if (cmd === "git" && args[0] === "-C" && args[2] === "rev-parse") {
+        return commandResult(0, "web-head");
+      }
+      // The check tool self-skips on missing deps and exits 0.
+      return commandResult(
+        0,
+        "Yarn checks were blocked because dependencies are not installed",
+      );
+    });
+
+    const result = await runPrePrChecksWithFixes(
+      "sbx-test-123",
+      { repositories: [config.repositories[0]!] },
+      "codex",
+      "gpt-5",
+      0,
+    );
+
+    expect(result.outcome).toBe("failed");
+    expect(result.passed).toBe(false);
+    expect(result.failures).toHaveLength(1);
+    expect(result.failures[0]).toMatchObject({
+      provider: "github",
+      repoPath: "acme/web",
+      command: "pnpm typecheck",
+    });
+    expect(result.summary.toLowerCase()).toContain("did not actually run");
+  });
+
   it("treats inability to inspect a repository as an execution failure", async () => {
     mockRunCommand.mockImplementation((cmd, args) => {
       if (cmd === "cat" && args[0] === WORKSPACE_MANIFEST_PATH) {

apps/worker/src/pre-pr-checks/runner.ts

@@ -179,14 +179,26 @@ async function runConfiguredPrePrChecks(
         command,
         exitCode: result.exitCode,
       });
-      if (result.exitCode !== 0) {
+      const stdout = await commandStdout(result);
+      const stderr = await commandStderr(result);
+      // A configured check that exits 0 while reporting that it never ran (its
+      // dependencies are not installed) must fail loudly instead of being trusted
+      // as a pass. The Run Workspace is never dependency-installed, so a check tool
+      // that self-skips on missing deps with a success exit code once let a blocked
+      // check clear the pre-PR gate: the branch was pushed and the PR's own CI then
+      // caught the lint failure the gate exists to prevent.
+      const blockedByMissingDependencies =
+        result.exitCode === 0 && checkBlockedByMissingDependencies(stdout, stderr);
+      if (result.exitCode !== 0 || blockedByMissingDependencies) {
         failures.push({
           provider: repo.provider,
           repoPath: repo.repoPath,
           command,
           exitCode: result.exitCode,
-          stdout: await commandStdout(result),
-          stderr: await commandStderr(result),
+          stdout,
+          stderr: blockedByMissingDependencies
+            ? [stderr, MISSING_DEPENDENCY_FAILURE_REASON].filter(Boolean).join("\n")
+            : stderr,
         });
       }
     }
@@ -469,6 +481,28 @@ function repositoryKey(repo: Pick<WorkspaceRepo, "provider" | "repoPath">): stri
   return `${repo.provider}:${repo.repoPath}`;
 }
 
+const MISSING_DEPENDENCY_FAILURE_REASON =
+  "Pre-PR check exited 0 but its dependencies are not installed, so the check did not actually run.";
+
+/**
+ * True when a check tool reported it could not run because the project's
+ * dependencies are not installed (yarn/npm/pnpm all print a variant of this,
+ * and some exit 0 while doing so). Matched against the tool's own output so an
+ * exit-0 "success" that never executed the check is surfaced as a failure
+ * instead of silently certifying the workspace.
+ */
+function checkBlockedByMissingDependencies(stdout: string, stderr: string): boolean {
+  const haystack = `${stderr}\n${stdout}`.toLowerCase();
+  return (
+    haystack.includes("dependencies are not installed") ||
+    haystack.includes("dependencies must be installed") ||
+    haystack.includes("run `yarn install`") ||
+    haystack.includes("run 'yarn install'") ||
+    haystack.includes("run `npm install`") ||
+    haystack.includes("run `pnpm install`")
+  );
+}
+
 async function commandStdout(result: SandboxCommandResult): Promise<string> {
   return (await result.stdout()).trim();
 }

docs/plans/2026-08-18-watch-external-ci-to-green.md

@@ -0,0 +1,74 @@
+# Design note: watch the external PR/MR CI pipeline to green
+
+Status: proposal, not implemented. Recommendation: file as its own ticket.
+
+## Problem
+
+Today the workflow bot runs the dashboard-configured pre-PR checks inside the Run
+Workspace, then pushes the branch and opens the PR/MR. Once the PR is open, the
+provider (GitHub Actions / GitLab CI) runs its own pipeline. The bot never looks
+at that pipeline. When the provider pipeline fails (for reasons the in-sandbox
+pre-PR checks did not or could not catch), nobody feeds that failure back into a
+fix loop. A client hit exactly this: an MR opened, GitLab's `lint` /
+`Lint-Docker-Frontend` jobs failed, and the run had already reported success.
+
+The related fix in this PR stops one silent-skip (a pre-PR check that exits 0
+while its dependencies are not installed now fails loudly). Watching the external
+pipeline is the broader, separate capability the client also asked for and is
+intentionally out of scope here.
+
+## Proposed behavior
+
+After `open_pr` publishes a PR/MR, a new optional block (working name
+`watch_ci_pipeline`) polls the provider's pipeline for the pushed head SHA until
+it reaches a terminal state, then branches on the result:
+
+1. Resolve the pipeline for the PR's head SHA via the VCS adapter
+   (`apps/worker/src/adapters/vcs/*`): GitHub check runs / commit statuses for a
+   ref; GitLab pipelines + jobs for a ref.
+2. Poll on an interval with a bounded deadline, driven by the run budget's
+   remaining duration (reuse `ctx.observeBudget()` so a stuck pipeline cannot
+   outlive the run). Poll cadence and max wait are block params.
+3. Ignore non-gating checks. Meticulous is explicitly excluded (it posts its own
+   visual-review status that must not gate the bot). Maintain an ignore-list of
+   check/job names (and/or contexts), defaulting to a Meticulous matcher, so the
+   watcher only waits on and reacts to gating jobs.
+4. Terminal outcomes:
+   - all gating jobs succeeded -> `ok: true` (branch to done / Slack / ticket move);
+   - one or more gating jobs failed -> collect each failed job's name + log tail
+     and feed them into the existing fix loop (same shape the pre-PR runner uses
+     for `runFixAgent`: a prompt with the failing job output, then re-push and
+     re-watch), capped by a max-attempts param and the run budget;
+   - pipeline never reaches terminal state before the deadline -> surface a
+     bounded, transparent failure (budget/deadline), never a silent pass.
+
+## Where it plugs in
+
+- New block type registered in `apps/worker/src/workflow-definition/block-registry.ts`
+  and a matching case in the interpreter/agent block switch (`agent.ts`).
+- New VCS adapter methods: `getPipelineForRef(headSha)` / `listCheckRunsForRef`
+  returning a normalized `{ name, status, conclusion, logsUrl }[]` across GitHub
+  and GitLab, plus a way to fetch a failed job's log tail.
+- Reuse the fix-agent machinery already in `pre-pr-checks/runner.ts`
+  (`runFixAgent`, `buildFixPrompt`) so external-CI failures and pre-PR failures
+  share one repair path and one budget accounting.
+- Ignore-list config (Meticulous by default) alongside the pre-PR checks config
+  so it is dashboard-managed and versioned like the rest of the gate config.
+
+## Risks / open questions (for the ticket)
+
+- Polling long pipelines against the per-invocation limit: this must be
+  budget-bounded and heartbeat-safe; a pipeline that runs for an hour cannot pin
+  the run. Confirm the WDK invocation model tolerates the poll loop.
+- Re-push after a fix re-triggers the provider pipeline, which re-triggers the
+  watcher: needs a clear max-attempts and loop-boundary so it cannot ping-pong.
+- Which statuses gate (required checks only vs all): should read the provider's
+  branch-protection / required-checks set where available rather than guessing.
+- Meticulous-style checks that are `pending` forever must be treated as ignored,
+  not as "still running", or the watcher waits on them until the deadline.
+
+## Recommendation
+
+File as its own ticket (its own block type, adapter surface, and budget/loop
+design). Do not fold it into the missing-dependency pre-PR fix, which is a
+narrow, self-contained correctness fix.