Skip to content

Commit 790ab12

Browse files
committed
docs(e2e): clarify reliability report inspection
Signed-off-by: Ho Lim <subhoya@gmail.com>
1 parent 9ceb96b commit 790ab12

1 file changed

Lines changed: 14 additions & 12 deletions

File tree

test/e2e/README.md

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -582,23 +582,25 @@ graph as the live targets:
582582
- GitHub Actions run history is the authoritative record for push and
583583
manual E2E results.
584584
- `E2E / Main Retry` publishes an advisory same-commit reliability table for
585-
trusted-main and explicit manual qualification runs. It keeps first-pass
586-
success, pass-after-retry, exhausted retries, and pass/fail flips distinct,
585+
trusted pushes to `main` and explicit manual qualification runs. It keeps
586+
first-pass success, pass-after-retry, exhausted retries, and pass/fail flips
587+
distinct,
587588
and never treats retry records or runner-pressure classifications as proof
588589
that a manual run reached a terminal result. Manual identity comes from the
589590
run-bound dispatch receipt; its terminal result additionally requires at
590591
least one canonical `nemoclaw-e2e-evidence-v1` manifest bound to the same run,
591592
attempt, candidate SHA, trusted workflow repository, workflow SHA, and job
592-
status. Trusted-main outcomes instead require the canonical retry-controller
593-
artifact. Missing or malformed identity/outcome evidence leaves a run
594-
unclassified. Retry and runner-pressure files contribute only fixed failure
595-
classes: their complete, missing, or malformed state is reported separately,
596-
so malformed classification data cannot erase an otherwise authenticated
597-
outcome. Both evidence-state counts appear in the grouped JSON and Markdown
598-
table. The table never changes a required check, release conclusion, or rerun
599-
decision.
600-
- The report job appends the Markdown table to the GitHub Actions job summary
601-
and uploads the bounded, allowlisted JSON and Markdown files as
593+
status. Outcomes from trusted pushes to `main` instead require the canonical
594+
retry-controller artifact. Missing or malformed identity/outcome evidence
595+
leaves a run unclassified. Retry and runner-pressure files contribute only
596+
allowlisted failure classes: their complete, missing, or malformed state is
597+
reported separately, so malformed classification data cannot erase an
598+
otherwise authenticated outcome. Both evidence-state counts appear in the
599+
grouped JSON and Markdown table. The table never changes a required check,
600+
release conclusion, or rerun decision.
601+
- The `report-same-commit-reliability` job appends the Markdown table to its
602+
GitHub Actions job summary and uploads the bounded, allowlisted
603+
`same-commit-reliability.json` and `same-commit-reliability.md` files as
602604
`same-commit-reliability-<source-run-id>-<attempt>` with 14-day retention.
603605
Artifact ZIP entries are structurally validated before an allowlisted file is
604606
read: ambiguous relative paths, links, encryption, split/ZIP64 archives,

0 commit comments

Comments
 (0)