Commit b2d1ce5
authored
fix(e2e): bind PR runs to managed image cohort (#9484)
## Summary
Manual PR E2E could test a candidate CLI against a released
managed-image cohort, so candidate/runtime revision skew could produce
the trusted-private OpenClaw MCP discovery failure reported in #8746.
This change binds affected same-repository PR runs to one exact
all-agent publication cohort and adds two required, credential-free
executions of the complete reported OpenClaw MCP bridge scenario before
merge.
## Related Issue
Fixes #8746
Related: #9464
## Changes
- Resolve the unique successful `Images / Managed Images` run for the
exact PR head from the trusted base workflow before candidate checkout.
- Download each immutable managed-image contract artifact by ID, verify
its digest and producer identity, and reject missing, mixed-revision,
mixed-release, mixed-cohort, or incomplete catalogs.
- Package only the verified catalog with the exact candidate CLI, after
removing any candidate-created catalog at the trust boundary.
- Make live onboarding and rebuild consume that exact catalog and reject
conflicts with legacy environment or durable receipt authority.
- Add two independent `PR exact OpenClaw trusted-private MCP discovery`
matrix jobs that run the unchanged `test/e2e/live/mcp-bridge.test.ts`
OpenClaw shard on fresh runners and sandboxes using the exact PR cohort.
- Preserve released-catalog behavior for external forks and PRs outside
the managed-image workflow path filter.
## Investigation Result and Ownership
The cutover recurrence recorded an empty fake-fixture request ledger
while trusted-private pins matched, placing the failure before the
adapter request rather than during a successful MCP protocol exchange.
Manual PR E2E had no exact-candidate publication input and could pair
the candidate CLI with the released managed-image catalog. The resulting
candidate/runtime revision skew is owned by NemoClaw CI orchestration,
not by the OpenShell credential, policy, or MCP protocol path.
This PR fixes that NemoClaw-owned cause by making the trusted base
resolve one immutable, exact-head publication cohort before candidate
code runs and by making live onboarding consume only that validated
catalog. It does not add a runtime workaround, timing retry,
credential-isolation exception, policy bypass, or OpenShell change. The
unchanged complete scenario must pass twice on the coherent cohort; a
recurrence of the empty-ledger signature blocks acceptance.
## Trust-Boundary Rationale
- The manual E2E planner executes the trusted workflow revision before
candidate checkout. Its exact read-only `actions`, `contents`, and
`pull-requests` permissions are required to bind the PR, workflow run,
and immutable artifacts; `GITHUB_TOKEN` is scoped to that resolver step,
candidate checkout does not persist credentials, and any
candidate-created catalog is removed before the trusted catalog is
installed.
- Producer and consumer jobs independently validate artifacts because
the artifact transfer is a trust boundary. They now invoke the same
TypeScript catalog-contract owner, so validation is repeated without
duplicating schema or authority.
- `E2E_MANAGED_IMAGE_REVISION` remains the released-catalog assertion
for unchanged non-PR qualification. The PR catalog path is separate and
simultaneous use fails closed; migrating every legacy E2E is outside
this fix.
- The credential-free MCP job uses the repository's existing official
cloudflared release pin, exact SHA-256, and Debian package-metadata
checks already enforced by the trusted MCP and standard-profile workflow
boundaries.
## Acceptance Criteria
- [ ] Both exact-head `Images / Managed Images / PR exact OpenClaw
trusted-private MCP discovery` matrix checks pass independently (pass 1
and pass 2); neither is a retry for the other.
- [ ] Each pass proves its catalog contains all three shipped agents at
revision `6a6406e44aec19d5178c35c9729f8240ceade0a4`, platform
`linux/amd64`, and one publication cohort before onboarding.
- [ ] Each pass reaches the same trusted-private discovery assertion
reported in #8746 and records `toolDiscovery={ok:true,count:2}` with
`fake_echo` and `fake_status`.
- [ ] Each pass records sanitized fixture requests for `initialize` and
`tools/list`, session-metadata presence, successful response status, and
matched synthetic credential rewriting.
- [ ] Each pass preserves provider attachment, policy pinning, adapter
status, credential isolation, DNS-rebinding denial, artifact credential
scan, and verified resource cleanup.
- [ ] The tests use no production credentials.
- [ ] All required CI, PR Review Advisor results, CodeRabbit review,
human review, unresolved-thread, mergeability, and exact-head
commit-verification gates are green before admin merge.
Evidence: pending for merge head
`6a6406e44aec19d5178c35c9729f8240ceade0a4`; the prior-head evidence was
superseded by the required `main` merge.
## Type of Change
- [ ] Code change (feature, bug fix, or refactor)
- [x] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)
## Quality Gates
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [x] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: fail-closed
trust-boundary review covers immutable artifact identity, exact producer
revision/cohort, candidate-code removal before trusted catalog install,
symlink-safe bounded catalog reads, no production credentials, and
fixture-secret artifact scanning.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:
## DGX Station Hardware Evidence
- [ ] Tested on DGX Station
- Tested commit: Not applicable
- Station profile/scenario: Not applicable
- Result: Not applicable
- Supporting evidence: Not applicable
## Verification
- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — 123 focused resolver,
workflow-contract, onboarding, rebuild, and artifact-integrity tests
passed before the required `main` merge. The only conflict was the retry
inventory; conflict-marker/diff checks and markdown lint passed, with
broader local reruns intentionally omitted per maintainer direction.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — awaiting exact-head PR CI.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
---
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Reliability**
* Managed-image catalogs are now resolved, validated, and installed with
exact revisions and secure file handling.
* Onboarding and workload rebuilds verify catalog consistency before
proceeding, reducing mismatched or incomplete image setups.
* **Testing**
* Added comprehensive validation for managed-image publication, artifact
integrity, catalog assembly, and trusted private MCP bridge workflows.
* **Documentation**
* Expanded end-to-end testing guidance covering catalog verification,
credential protection, diagnostics, and required acceptance runs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->1 parent ed7378f commit b2d1ce5
16 files changed
Lines changed: 1102 additions & 33 deletions
File tree
- .github/workflows
- src/lib/onboard
- managed-workload
- workload
- test
- e2e
- support
- helpers
- tools/e2e
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
172 | 172 | | |
173 | 173 | | |
174 | 174 | | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
175 | 179 | | |
176 | 180 | | |
177 | 181 | | |
| |||
536 | 540 | | |
537 | 541 | | |
538 | 542 | | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
539 | 553 | | |
540 | 554 | | |
541 | 555 | | |
| |||
652 | 666 | | |
653 | 667 | | |
654 | 668 | | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
655 | 677 | | |
656 | 678 | | |
657 | 679 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
863 | 863 | | |
864 | 864 | | |
865 | 865 | | |
| 866 | + | |
866 | 867 | | |
867 | 868 | | |
868 | 869 | | |
869 | | - | |
870 | | - | |
871 | | - | |
872 | | - | |
873 | | - | |
874 | | - | |
875 | | - | |
876 | | - | |
877 | | - | |
878 | | - | |
879 | | - | |
880 | | - | |
881 | | - | |
882 | | - | |
883 | | - | |
884 | | - | |
885 | | - | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
886 | 875 | | |
887 | 876 | | |
888 | | - | |
889 | | - | |
890 | | - | |
| 877 | + | |
891 | 878 | | |
892 | 879 | | |
893 | 880 | | |
| |||
904 | 891 | | |
905 | 892 | | |
906 | 893 | | |
907 | | - | |
| 894 | + | |
908 | 895 | | |
909 | 896 | | |
910 | 897 | | |
| |||
916 | 903 | | |
917 | 904 | | |
918 | 905 | | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
| 985 | + | |
| 986 | + | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
| 991 | + | |
| 992 | + | |
| 993 | + | |
| 994 | + | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
| 1001 | + | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
| 1009 | + | |
| 1010 | + | |
| 1011 | + | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
919 | 1034 | | |
920 | 1035 | | |
921 | 1036 | | |
| |||
Lines changed: 29 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
4 | 8 | | |
5 | 9 | | |
6 | 10 | | |
| |||
104 | 108 | | |
105 | 109 | | |
106 | 110 | | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
107 | 136 | | |
108 | 137 | | |
109 | 138 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
51 | 51 | | |
52 | 52 | | |
53 | 53 | | |
| 54 | + | |
54 | 55 | | |
55 | 56 | | |
56 | 57 | | |
| |||
196 | 197 | | |
197 | 198 | | |
198 | 199 | | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
199 | 204 | | |
200 | 205 | | |
201 | 206 | | |
| |||
210 | 215 | | |
211 | 216 | | |
212 | 217 | | |
213 | | - | |
| 218 | + | |
| 219 | + | |
214 | 220 | | |
215 | 221 | | |
216 | 222 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
85 | 86 | | |
86 | 87 | | |
87 | 88 | | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
88 | 150 | | |
89 | 151 | | |
90 | 152 | | |
| |||
122 | 184 | | |
123 | 185 | | |
124 | 186 | | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
125 | 204 | | |
126 | 205 | | |
127 | 206 | | |
| |||
0 commit comments