-
Notifications
You must be signed in to change notification settings - Fork 1
173 lines (166 loc) · 9.26 KB
/
Copy pathtriage-dispatch.yml
File metadata and controls
173 lines (166 loc) · 9.26 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
# Human-gated GitHub automation for the daily triage.
# Phase 1 (propose): after the daily-stable run finishes red, run the triage
# skill non-interactively and post the plan as a comment on the umbrella issue.
# Creates nothing on its own. (workflow_run is used, NOT issues.opened: the
# umbrella issue is created by daily-stable's GITHUB_TOKEN, and GitHub suppresses
# workflow runs for GITHUB_TOKEN-raised events — an issues trigger would never
# fire. workflow_run is exempt from that suppression.)
# Phase 2 (execute): when a triage-team member replies "pode abrir" on that
# umbrella, run the skill again to create/enrich the dedicated issues.
# No code is mutated from CI: contents is read-only, so @stable removal stays a
# manual/local PR (the skill's 2nd gate); allowedTools is Read,Bash.
# NOTE: workflow_run / issue_comment triggers only fire from the workflow file
# on the DEFAULT branch — the auto-triggers are live only after this merges to
# main. Pre-merge, use the workflow_dispatch entry to test `propose` on a branch.
name: Triage Dispatch
on:
workflow_run:
workflows: ["Daily Stable E2E"]
types: [completed]
issue_comment:
types: [created]
workflow_dispatch:
inputs:
issue:
description: "Umbrella issue number to (re)propose against"
required: true
# id-token: write is REQUIRED by anthropics/claude-code-action@v1 — it mints an
# OIDC token to authenticate; without it the action fails fast with
# "Could not fetch an OIDC token". It grants no repo write: contents stays read,
# so @stable removal remains a manual/local PR.
permissions:
issues: write
contents: read
id-token: write
# actions: read lets the propose job download the triggering daily run's
# results.json artifact (playwright-json-daily-<run_id>) so Phase 5 can read
# per-skip reasons instead of only the history file's skip total (#791).
actions: read
jobs:
propose:
name: Propose triage plan
# workflow_dispatch is gated on a trusted actor: without this, any collaborator
# who can dispatch a workflow could run the triage skill against an arbitrary
# issue number. The workflow_run path needs no actor guard — it only fires from
# the real Daily Stable E2E run and is already narrowed to schedule + failure.
if: >-
(github.event_name == 'workflow_dispatch' &&
contains(fromJSON('["rafaelgiln","Victor-w-Madeira","daniellicnerski1"]'), github.actor)) ||
(github.event_name == 'workflow_run' &&
github.event.workflow_run.conclusion == 'failure' &&
github.event.workflow_run.event == 'schedule')
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v7
# Best-effort: pull the triggering daily's results.json so the skill's
# INTAKE can pass --results and Phase 5 (SKIPS) reads real per-skip
# reasons. Only the workflow_run path has a source run; a manual
# workflow_dispatch has none, so it runs history-only (as before). Never
# fails the job — if the artifact is missing/expired, propose degrades to
# history-only and says so in the proposal.
- name: Download triggering daily results.json (best-effort)
if: github.event_name == 'workflow_run'
continue-on-error: true
env:
GH_TOKEN: ${{ github.token }}
DAILY_RUN_ID: ${{ github.event.workflow_run.id }}
run: |
gh run download "$DAILY_RUN_ID" --repo "$GITHUB_REPOSITORY" \
-n "playwright-json-daily-$DAILY_RUN_ID" -D . \
|| echo "::warning::results.json artifact unavailable for run $DAILY_RUN_ID — propose runs history-only."
if [ -f results.json ]; then echo "results.json present ($(wc -c < results.json) bytes)."; else echo "no results.json — history-only."; fi
# Do not start the agent when the triggering run produced nothing to triage
# (#1178). Until this gate existed, the step above degraded to
# "history-only" and the agent ran anyway: on 2026-07-31 the daily's four
# shards all failed without a blob, the merge job died on `No report files
# found in .../all-blobs`, no results.json was uploaded, and the agent spent
# 28 of its 30 turns and $1.66 (its own log reports `num_turns` and
# `total_cost_usd`) reconstructing a run that had recorded ZERO test results.
#
# The decision reuses `analyze()` from scripts/check-run-integrity.mjs — the
# daily's own runguard logic (#1012) — so the two can never disagree, and so
# the case that makes a naive test-count check wrong is handled: a PARTIAL
# run, where some shards aborted in globalSetup while others executed
# (#1058), still RUNS the agent. Skipping is loud (::warning:: + a run-summary
# line), never silent; an unparseable-but-present report fails the step
# rather than reading as "nothing to triage" (#1035).
- name: Gate — does the triggering run give the agent anything to triage?
id: triage_input
run: node scripts/triage-input-gate.mjs --results results.json
# The gate binds the AUTOMATIC path only. A `workflow_dispatch` is a human
# asking for this triage with an explicit `--issue`, and it downloads no
# artifact at all (there is no triggering run id), so gating it would refuse
# the one path that exists for re-proposing on demand — including the
# per-model A/B #1171 needs. Same reasoning as leaving `execute` ungated.
- uses: anthropics/claude-code-action@v1
if: >-
github.event_name == 'workflow_dispatch' ||
steps.triage_input.outputs.should_run_agent == 'true'
env:
GH_TOKEN: ${{ github.token }}
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
prompt: "/langflow-e2e-triage --phase propose ${{ inputs.issue && format('--issue {0}', inputs.issue) || '' }}"
# --model pinned (#1178): the action's default is `claude-opus-5[1m]`, and
# nothing here needs Opus-tier reasoning at Opus-tier price. Sonnet 5
# keeps the 1M context window this job depends on (it reads CI logs) at
# 3/5 of the per-token price — on the three measured runs, $4.71 → ~$2.80.
# NOT haiku: its window is 200K, and the most expensive runs are the ones
# with the largest input, so it would overflow exactly where it matters.
# Worth an A/B once the input is bounded — via `workflow_dispatch --issue N`
# on the same umbrella — and #1171 is the standing reminder that a cheaper
# model has to be validated, not assumed.
claude_args: '--allowedTools "Read,Bash" --max-turns 30 --model claude-sonnet-5'
execute:
name: Execute approved triage
if: >-
github.event_name == 'issue_comment' &&
github.event.issue.pull_request == null &&
github.event.issue.state == 'open' &&
contains(github.event.issue.labels.*.name, 'daily-failure') &&
startsWith(github.event.comment.body, 'pode abrir') &&
contains(fromJSON('["rafaelgiln","Victor-w-Madeira","daniellicnerski1"]'), github.event.comment.user.login)
runs-on: ubuntu-latest
timeout-minutes: 20
concurrency:
group: triage-execute-${{ github.event.issue.number }}
cancel-in-progress: false
steps:
- uses: actions/checkout@v7
# Stamped BEFORE the agent runs so the guard below can tell the issues this
# run created from every other open daily-failure issue. Minus a minute of
# slack for clock skew between the runner and the API.
- name: Mark the start of this run
id: started
run: echo "at=$(date -u -d '-1 minute' +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT"
- uses: anthropics/claude-code-action@v1
env:
GH_TOKEN: ${{ github.token }}
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
prompt: "/langflow-e2e-triage --phase execute --issue ${{ github.event.issue.number }}"
# Same pin as propose (#1178). NOT gated on a report: execute is triggered
# by a human "pode abrir" on a plan that already exists, so it has
# something to do regardless of what the daily uploaded.
claude_args: '--allowedTools "Read,Bash" --max-turns 40 --model claude-sonnet-5'
# The contract gate for the AGENT path (#1035).
#
# An `issues.opened` trigger cannot do this job: the step above creates the
# issues as GITHUB_TOKEN, and GitHub suppresses workflow runs for
# GITHUB_TOKEN-raised events (the same constraint that put this whole
# workflow on workflow_run). An issues-triggered gate would never fire here
# while looking green — so the check runs inline instead, outside the agent
# but inside its job.
#
# `if: always()` — a triage that died mid-way can still have created issues,
# and those are exactly the ones worth checking.
# `continue-on-error` — report, never block: a malformed issue still carries
# evidence someone needs, so it gets a comment and a label, not a red job.
- name: Guard — issues created by this run meet the dedicated-issue contract
if: always()
continue-on-error: true
uses: ./.github/actions/guard-dedicated-issue
with:
since: ${{ steps.started.outputs.at }}
github_token: ${{ github.token }}