Skip to content

Commit b2d1ce5

Browse files
authored
fix(e2e): bind PR runs to managed image cohort (#9484)
## Summary Manual PR E2E could test a candidate CLI against a released managed-image cohort, so candidate/runtime revision skew could produce the trusted-private OpenClaw MCP discovery failure reported in #8746. This change binds affected same-repository PR runs to one exact all-agent publication cohort and adds two required, credential-free executions of the complete reported OpenClaw MCP bridge scenario before merge. ## Related Issue Fixes #8746 Related: #9464 ## Changes - Resolve the unique successful `Images / Managed Images` run for the exact PR head from the trusted base workflow before candidate checkout. - Download each immutable managed-image contract artifact by ID, verify its digest and producer identity, and reject missing, mixed-revision, mixed-release, mixed-cohort, or incomplete catalogs. - Package only the verified catalog with the exact candidate CLI, after removing any candidate-created catalog at the trust boundary. - Make live onboarding and rebuild consume that exact catalog and reject conflicts with legacy environment or durable receipt authority. - Add two independent `PR exact OpenClaw trusted-private MCP discovery` matrix jobs that run the unchanged `test/e2e/live/mcp-bridge.test.ts` OpenClaw shard on fresh runners and sandboxes using the exact PR cohort. - Preserve released-catalog behavior for external forks and PRs outside the managed-image workflow path filter. ## Investigation Result and Ownership The cutover recurrence recorded an empty fake-fixture request ledger while trusted-private pins matched, placing the failure before the adapter request rather than during a successful MCP protocol exchange. Manual PR E2E had no exact-candidate publication input and could pair the candidate CLI with the released managed-image catalog. The resulting candidate/runtime revision skew is owned by NemoClaw CI orchestration, not by the OpenShell credential, policy, or MCP protocol path. This PR fixes that NemoClaw-owned cause by making the trusted base resolve one immutable, exact-head publication cohort before candidate code runs and by making live onboarding consume only that validated catalog. It does not add a runtime workaround, timing retry, credential-isolation exception, policy bypass, or OpenShell change. The unchanged complete scenario must pass twice on the coherent cohort; a recurrence of the empty-ledger signature blocks acceptance. ## Trust-Boundary Rationale - The manual E2E planner executes the trusted workflow revision before candidate checkout. Its exact read-only `actions`, `contents`, and `pull-requests` permissions are required to bind the PR, workflow run, and immutable artifacts; `GITHUB_TOKEN` is scoped to that resolver step, candidate checkout does not persist credentials, and any candidate-created catalog is removed before the trusted catalog is installed. - Producer and consumer jobs independently validate artifacts because the artifact transfer is a trust boundary. They now invoke the same TypeScript catalog-contract owner, so validation is repeated without duplicating schema or authority. - `E2E_MANAGED_IMAGE_REVISION` remains the released-catalog assertion for unchanged non-PR qualification. The PR catalog path is separate and simultaneous use fails closed; migrating every legacy E2E is outside this fix. - The credential-free MCP job uses the repository's existing official cloudflared release pin, exact SHA-256, and Debian package-metadata checks already enforced by the trusted MCP and standard-profile workflow boundaries. ## Acceptance Criteria - [ ] Both exact-head `Images / Managed Images / PR exact OpenClaw trusted-private MCP discovery` matrix checks pass independently (pass 1 and pass 2); neither is a retry for the other. - [ ] Each pass proves its catalog contains all three shipped agents at revision `6a6406e44aec19d5178c35c9729f8240ceade0a4`, platform `linux/amd64`, and one publication cohort before onboarding. - [ ] Each pass reaches the same trusted-private discovery assertion reported in #8746 and records `toolDiscovery={ok:true,count:2}` with `fake_echo` and `fake_status`. - [ ] Each pass records sanitized fixture requests for `initialize` and `tools/list`, session-metadata presence, successful response status, and matched synthetic credential rewriting. - [ ] Each pass preserves provider attachment, policy pinning, adapter status, credential isolation, DNS-rebinding denial, artifact credential scan, and verified resource cleanup. - [ ] The tests use no production credentials. - [ ] All required CI, PR Review Advisor results, CodeRabbit review, human review, unresolved-thread, mergeability, and exact-head commit-verification gates are green before admin merge. Evidence: pending for merge head `6a6406e44aec19d5178c35c9729f8240ceade0a4`; the prior-head evidence was superseded by the required `main` merge. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: fail-closed trust-boundary review covers immutable artifact identity, exact producer revision/cohort, candidate-code removal before trusted catalog install, symlink-safe bounded catalog reads, no production credentials, and fixture-secret artifact scanning. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable - Station profile/scenario: Not applicable - Result: Not applicable - Supporting evidence: Not applicable ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 123 focused resolver, workflow-contract, onboarding, rebuild, and artifact-integrity tests passed before the required `main` merge. The only conflict was the retry inventory; conflict-marker/diff checks and markdown lint passed, with broader local reruns intentionally omitted per maintainer direction. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — awaiting exact-head PR CI. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Reliability** * Managed-image catalogs are now resolved, validated, and installed with exact revisions and secure file handling. * Onboarding and workload rebuilds verify catalog consistency before proceeding, reducing mismatched or incomplete image setups. * **Testing** * Added comprehensive validation for managed-image publication, artifact integrity, catalog assembly, and trusted private MCP bridge workflows. * **Documentation** * Expanded end-to-end testing guidance covering catalog verification, credential protection, diagnostics, and required acceptance runs. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent ed7378f commit b2d1ce5

16 files changed

Lines changed: 1102 additions & 33 deletions

.github/workflows/e2e.yaml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,10 @@ jobs:
172172
generate-matrix:
173173
runs-on: ubuntu-latest
174174
timeout-minutes: 10
175+
permissions:
176+
actions: read
177+
contents: read
178+
pull-requests: read
175179
outputs:
176180
cli_artifact_provenance: ${{ steps.record_cli_artifact.outputs.provenance }}
177181
e2e_credentials_allowed: ${{ steps.e2e_credentials.outputs.allowed }}
@@ -536,6 +540,16 @@ jobs:
536540
fi
537541
fi
538542
543+
- name: Resolve exact PR managed-image catalog
544+
if: ${{ inputs.checkout_sha != '' && (inputs.jobs != 'native-runtime-qualification-producer' || inputs.targets != '') }}
545+
env:
546+
BASE_SHA: ${{ inputs.base_sha }}
547+
CANDIDATE_REPOSITORY: ${{ inputs.checkout_repository }}
548+
CANDIDATE_SHA: ${{ inputs.checkout_sha }}
549+
GITHUB_TOKEN: ${{ github.token }}
550+
PR_NUMBER: ${{ inputs.pr_number }}
551+
run: node --experimental-strip-types --no-warnings tools/e2e/pr-managed-image-publication.mts "${RUNNER_TEMP}/pr-managed-image-catalog.json"
552+
539553
- name: Check out E2E candidate
540554
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
541555
if: ${{ inputs.checkout_sha == '' || inputs.jobs != 'native-runtime-qualification-producer' || inputs.targets != '' }}
@@ -652,6 +666,14 @@ jobs:
652666
' dist/build-identity.json >/dev/null ||
653667
{ echo "::error::candidate CLI build identity does not match the candidate commit SHA"; exit 1; }
654668
669+
managed_catalog="${RUNNER_TEMP}/pr-managed-image-catalog.json"
670+
rm -f -- dist/e2e-managed-image-catalog.json
671+
if [[ -e "$managed_catalog" ]]; then
672+
[[ -f "$managed_catalog" && ! -L "$managed_catalog" && -s "$managed_catalog" ]] ||
673+
{ echo "::error::trusted PR managed-image catalog is not a nonempty regular file"; exit 1; }
674+
install -m 0600 "$managed_catalog" dist/e2e-managed-image-catalog.json
675+
fi
676+
655677
artifact_dir="${RUNNER_TEMP}/nemoclaw-cli-artifact"
656678
install -d -m 0700 "$artifact_dir"
657679
payload="$artifact_dir/nemoclaw-cli.tar"

.github/workflows/managed-images.yaml

Lines changed: 136 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -863,31 +863,18 @@ jobs:
863863
shell: bash
864864
run: |
865865
set -euo pipefail
866+
npm ci --ignore-scripts
866867
mapfile -d '' contracts < <(
867868
find "$RUNNER_TEMP/managed-pr-contracts" -type f -name contract.json -print0
868869
)
869-
[[ "${#contracts[@]}" -eq 3 ]] || {
870-
echo "ERROR: exact all-agent activation requires three contracts" >&2
871-
exit 1
872-
}
873-
jq -se \
874-
--arg revision "$CANDIDATE_SHA" '
875-
if (
876-
length == 3 and
877-
([.[].agent] | sort) == ["hermes", "langchain-deepagents-code", "openclaw"] and
878-
([.[].source.revision] | unique) == [$revision] and
879-
([.[].source.cohort] | unique | length) == 1 and
880-
([.[].platform] | unique) == ["linux/amd64"]
881-
)
882-
then map({key: .agent, value: .}) | from_entries
883-
else error("published PR contracts do not form one exact all-agent cohort")
884-
end
885-
' "${contracts[@]}" > "$NEMOCLAW_MANAGED_ACTIVATION_CATALOG"
870+
node --experimental-strip-types --no-warnings \
871+
tools/e2e/pr-managed-image-publication.mts assemble \
872+
"$CANDIDATE_SHA" \
873+
"$NEMOCLAW_MANAGED_ACTIVATION_CATALOG" \
874+
"${contracts[@]}"
886875
887876
- name: Build exact candidate CLI
888-
run: |
889-
npm ci --ignore-scripts
890-
npm run build:cli
877+
run: npm run build:cli
891878

892879
- name: Install OpenShell CLI
893880
shell: bash
@@ -904,7 +891,7 @@ jobs:
904891
export PATH="$HOME/.local/bin:$HOME/.npm-global/bin:$PATH"
905892
export OPENSHELL_BIN="$(command -v openshell)"
906893
"$OPENSHELL_BIN" --version
907-
npx tsx tools/e2e/live-vitest-invocation.mts run \
894+
npx --no-install tsx tools/e2e/live-vitest-invocation.mts run \
908895
--test-path test/e2e/live/managed-image-activation-e2e.test.ts
909896
910897
- name: Upload managed runtime activation evidence
@@ -916,6 +903,134 @@ jobs:
916903
if-no-files-found: error
917904
retention-days: 1
918905

906+
pr-openclaw-mcp-discovery:
907+
name: PR exact OpenClaw trusted-private MCP discovery (pass ${{ matrix.pass }})
908+
needs: pr-build-and-entrypoint
909+
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
910+
runs-on: ubuntu-24.04
911+
timeout-minutes: 60
912+
permissions:
913+
contents: read
914+
strategy:
915+
fail-fast: false
916+
matrix:
917+
pass: [1, 2]
918+
env:
919+
CANDIDATE_SHA: ${{ github.event.pull_request.head.sha }}
920+
E2E_ARTIFACT_DIR: ${{ github.workspace }}/e2e-artifacts/live/pr-exact-mcp/pass-${{ matrix.pass }}
921+
E2E_JOB: "1"
922+
E2E_TARGET_ID: mcp-bridge
923+
NEMOCLAW_CLI_BIN: ${{ github.workspace }}/bin/nemoclaw.js
924+
NEMOCLAW_E2E_EXPECTED_SHA: ${{ github.event.pull_request.head.sha }}
925+
NEMOCLAW_E2E_MANAGED_IMAGE_CATALOG: ${{ github.workspace }}/managed-pr-catalog.json
926+
NEMOCLAW_E2E_REQUIRE_EXECUTED_TEST: "1"
927+
NEMOCLAW_E2E_SHARD: openclaw
928+
NEMOCLAW_MCP_BRIDGE_AGENT: openclaw
929+
NEMOCLAW_NON_INTERACTIVE: "1"
930+
NEMOCLAW_OPENSHELL_EXACT_MAIN_PROOF: "1"
931+
NEMOCLAW_RUN_LIVE_E2E: "1"
932+
NEMOCLAW_SANDBOX_NAME: e2e-pr-exact-mcp-${{ matrix.pass }}
933+
OPENSHELL_DOCKER_SUPERVISOR_IMAGE: ghcr.io/nvidia/openshell/supervisor@sha256:b58be5e40c788977ffa0e8305a8cad9c656efdf1a3fe182582a00ca870bb0edb
934+
steps:
935+
- name: Checkout exact PR head
936+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
937+
with:
938+
ref: ${{ github.event.pull_request.head.sha }}
939+
persist-credentials: false
940+
941+
- name: Set up Node.js
942+
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
943+
with:
944+
node-version: 22.19.0
945+
946+
- name: Bind E2E correlation identity
947+
shell: bash
948+
run: |
949+
set -euo pipefail
950+
correlation_id="$(node --input-type=module -e \
951+
'import { randomUUID } from "node:crypto"; console.log(randomUUID())')"
952+
[[ "$correlation_id" =~ ^[a-f0-9]{8}-[a-f0-9]{4}-4[a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$ ]]
953+
printf 'NEMOCLAW_E2E_CORRELATION_ID=%s\n' "$correlation_id" >> "$GITHUB_ENV"
954+
955+
- name: Download exact published all-agent contracts
956+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
957+
with:
958+
pattern: managed-pr-contract-${{ github.run_id }}-${{ github.run_attempt }}-*
959+
path: ${{ runner.temp }}/managed-pr-contracts
960+
merge-multiple: false
961+
962+
- name: Assemble exact all-agent MCP catalog
963+
shell: bash
964+
run: |
965+
set -euo pipefail
966+
npm ci --ignore-scripts
967+
mapfile -d '' contracts < <(
968+
find "$RUNNER_TEMP/managed-pr-contracts" -type f -name contract.json -print0
969+
)
970+
node --experimental-strip-types --no-warnings \
971+
tools/e2e/pr-managed-image-publication.mts assemble \
972+
"$CANDIDATE_SHA" \
973+
"$NEMOCLAW_E2E_MANAGED_IMAGE_CATALOG" \
974+
"${contracts[@]}"
975+
976+
- name: Build exact candidate CLI
977+
run: npm run build:cli
978+
979+
- name: Install and verify cloudflared prerequisite
980+
env:
981+
CLOUDFLARED_VERSION: "2026.6.1"
982+
CLOUDFLARED_DEB_SHA256: "ccd02ec216c62bfa573395d8f72cb2e91e95cbdf8726a8acc06b3e2d9aa31526"
983+
run: |
984+
set -euo pipefail
985+
cloudflared_deb="${RUNNER_TEMP}/cloudflared-${CLOUDFLARED_VERSION}-linux-amd64.deb"
986+
curl -fL "https://github.com/cloudflare/cloudflared/releases/download/${CLOUDFLARED_VERSION}/cloudflared-linux-amd64.deb" -o "${cloudflared_deb}"
987+
printf '%s %s\n' "${CLOUDFLARED_DEB_SHA256}" "${cloudflared_deb}" | sha256sum -c -
988+
package="$(dpkg-deb -f "${cloudflared_deb}" Package)"
989+
version="$(dpkg-deb -f "${cloudflared_deb}" Version)"
990+
architecture="$(dpkg-deb -f "${cloudflared_deb}" Architecture)"
991+
if [[ "${package}" != "cloudflared" || "${version}" != "${CLOUDFLARED_VERSION}" || "${architecture}" != "amd64" ]]; then
992+
printf 'Unexpected cloudflared package metadata: package=%s version=%s architecture=%s\n' "${package}" "${version}" "${architecture}" >&2
993+
exit 1
994+
fi
995+
sudo dpkg -i "${cloudflared_deb}"
996+
cloudflared --version | grep -F "cloudflared version ${CLOUDFLARED_VERSION}"
997+
998+
- name: Generate MCP test TLS
999+
run: bash test/e2e/setup-mcp-test-tls.sh
1000+
1001+
- name: Install OpenShell CLI
1002+
env:
1003+
NEMOCLAW_OPENSHELL_FORCE_INSTALL: "1"
1004+
run: bash scripts/install-openshell.sh
1005+
1006+
- name: Run exact OpenClaw trusted-private MCP discovery
1007+
shell: bash
1008+
run: |
1009+
set -euo pipefail
1010+
[[ "$(git rev-parse --verify HEAD)" == "$CANDIDATE_SHA" ]] || {
1011+
echo "ERROR: MCP discovery checkout does not match the exact PR head" >&2
1012+
exit 1
1013+
}
1014+
export PATH="$HOME/.local/bin:$HOME/.npm-global/bin:$PATH"
1015+
export OPENSHELL_BIN="$(command -v openshell)"
1016+
"$OPENSHELL_BIN" --version
1017+
npx --no-install tsx tools/e2e/live-vitest-invocation.mts run \
1018+
--test-path test/e2e/live/mcp-bridge.test.ts
1019+
1020+
- id: mcp_artifact_secret_scan
1021+
name: Scan MCP artifacts for fixture credentials
1022+
if: always()
1023+
run: npx --no-install tsx tools/e2e/assert-mcp-artifact-secrets-absent.mts "$E2E_ARTIFACT_DIR"
1024+
1025+
- name: Upload exact MCP discovery evidence
1026+
if: ${{ always() && steps.mcp_artifact_secret_scan.outcome == 'success' }}
1027+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
1028+
with:
1029+
name: pr-exact-openclaw-mcp-${{ github.run_id }}-${{ github.run_attempt }}-pass-${{ matrix.pass }}
1030+
path: e2e-artifacts/live/pr-exact-mcp/pass-${{ matrix.pass }}/
1031+
if-no-files-found: error
1032+
retention-days: 1
1033+
9191034
pi-candidate:
9201035
name: Build and validate the Pi candidate managed image (${{ matrix.arch }})
9211036
if: github.repository == 'NVIDIA/NemoClaw' && github.event_name == 'pull_request'

src/lib/onboard/managed-workload/onboard-orchestration.test.ts

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,10 @@
11
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
22
// SPDX-License-Identifier: Apache-2.0
33

4+
import fs from "node:fs";
5+
import os from "node:os";
6+
import path from "node:path";
7+
48
import { describe, expect, it, vi } from "vitest";
59

610
import { createHermesStateVolumeDockerHarness } from "../__test-helpers__/hermes-state-volume";
@@ -104,6 +108,31 @@ describe("managed workload onboard orchestration", () => {
104108
);
105109
});
106110

111+
it("binds fresh onboarding to the exact PR catalog (#9464)", async () => {
112+
const catalogRevision = "b".repeat(40);
113+
const fixtureRoot = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-live-e2e-catalog-"));
114+
const catalogPath = path.join(fixtureRoot, "catalog.json");
115+
fs.writeFileSync(catalogPath, "{}\n", { mode: 0o600 });
116+
try {
117+
const { prepared, runtime } = createFreshOnboardingRuntime({
118+
GITHUB_ACTIONS: "true",
119+
NEMOCLAW_RUN_LIVE_E2E: "1",
120+
NEMOCLAW_E2E_EXPECTED_SHA: catalogRevision,
121+
NEMOCLAW_E2E_MANAGED_IMAGE_CATALOG: catalogPath,
122+
});
123+
124+
await expect(runtime.ensurePreparedWorkload()).resolves.toBe(prepared);
125+
expect(prepareSandboxWorkloadSource).toHaveBeenCalledExactlyOnceWith(
126+
expect.objectContaining({
127+
catalogPath,
128+
expectedCatalogRevision: catalogRevision,
129+
}),
130+
);
131+
} finally {
132+
fs.rmSync(fixtureRoot, { force: true, recursive: true });
133+
}
134+
});
135+
107136
it("omits the qualification catalog revision outside GitHub Actions (#9385)", async () => {
108137
const { prepared, runtime } = createFreshOnboardingRuntime({
109138
E2E_MANAGED_IMAGE_REVISION: "a".repeat(40),

src/lib/onboard/managed-workload/onboard-orchestration.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ import {
5151
import { getSandboxReadyTimeoutSecs } from "../sandbox-gpu-create";
5252
import type { SandboxGpuConfig } from "../sandbox-gpu-mode";
5353
import {
54+
liveE2eManagedImageCatalog,
5455
liveE2eManagedImageRevision,
5556
type PreparedSandboxWorkloadSource,
5657
prepareSandboxWorkloadSource,
@@ -196,6 +197,10 @@ export function createManagedWorkloadOnboardRuntime(
196197

197198
const ensurePreparedWorkload = async (): Promise<PreparedSandboxWorkloadSource> => {
198199
const catalogRevision = liveE2eManagedImageRevision(input.startupProfile.environment);
200+
const liveCatalog = liveE2eManagedImageCatalog(input.startupProfile.environment);
201+
if (catalogRevision && liveCatalog) {
202+
throw new Error("live E2E managed-image revision and catalog authority conflict");
203+
}
199204
preparedWorkloadPromise ??= input.managedWorkloadRebuild
200205
? Promise.resolve(
201206
prepareSandboxWorkloadSourceFromRebuildHandoff(
@@ -210,7 +215,8 @@ export function createManagedWorkloadOnboardRuntime(
210215
customDockerfilePath: input.customDockerfilePath,
211216
runtime: runtimeCapabilities,
212217
version: getVersion({ rootDir: input.rootDir }),
213-
catalogPath: input.tempManagedRuntimeCatalog,
218+
catalogPath: input.tempManagedRuntimeCatalog ?? liveCatalog?.path ?? null,
219+
...(liveCatalog ? { expectedCatalogRevision: liveCatalog.revision } : {}),
214220
...(catalogRevision ? { catalogRevision } : {}),
215221
acceptedCandidateContract: isCandidateAgent(input.agentName)
216222
? readCandidateQualificationReceipt(input.agentName)

src/lib/onboard/sandbox-workload-preparation.test.ts

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ import {
2525
} from "./managed-image/contract";
2626
import { createRuntimeProviderBundleRegistry } from "./runtime-provider/registry";
2727
import {
28+
liveE2eManagedImageCatalog,
2829
prepareSandboxWorkloadSource,
2930
SandboxWorkloadPreparationError,
3031
} from "./workload/preparation";
@@ -85,6 +86,67 @@ function input(agentName: string) {
8586
}
8687

8788
describe("sandbox workload preparation", () => {
89+
it("selects an exact embedded catalog only for live PR E2E (#9464)", () => {
90+
const fixtureRoot = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-live-e2e-catalog-"));
91+
const catalogPath = path.join(fixtureRoot, "catalog.json");
92+
const packagedCatalogPath = path.join(fixtureRoot, "dist", "e2e-managed-image-catalog.json");
93+
fs.mkdirSync(path.dirname(packagedCatalogPath));
94+
fs.writeFileSync(catalogPath, "{}\n", { mode: 0o600 });
95+
fs.writeFileSync(packagedCatalogPath, "{}\n", { mode: 0o600 });
96+
try {
97+
expect(
98+
liveE2eManagedImageCatalog({
99+
GITHUB_ACTIONS: "true",
100+
NEMOCLAW_RUN_LIVE_E2E: "1",
101+
NEMOCLAW_E2E_EXPECTED_SHA: REVISION,
102+
NEMOCLAW_E2E_MANAGED_IMAGE_CATALOG: catalogPath,
103+
}),
104+
).toEqual({ path: catalogPath, revision: REVISION });
105+
expect(
106+
liveE2eManagedImageCatalog({
107+
GITHUB_ACTIONS: "true",
108+
GITHUB_WORKSPACE: fixtureRoot,
109+
NEMOCLAW_RUN_LIVE_E2E: "1",
110+
NEMOCLAW_E2E_EXPECTED_SHA: REVISION,
111+
}),
112+
).toEqual({ path: packagedCatalogPath, revision: REVISION });
113+
expect(
114+
liveE2eManagedImageCatalog({
115+
NEMOCLAW_RUN_LIVE_E2E: "1",
116+
NEMOCLAW_E2E_EXPECTED_SHA: REVISION,
117+
NEMOCLAW_E2E_MANAGED_IMAGE_CATALOG: catalogPath,
118+
}),
119+
).toBeNull();
120+
expect(
121+
liveE2eManagedImageCatalog({
122+
GITHUB_ACTIONS: "true",
123+
NEMOCLAW_RUN_LIVE_E2E: "1",
124+
NEMOCLAW_E2E_EXPECTED_SHA: REVISION,
125+
NEMOCLAW_E2E_MANAGED_IMAGE_CATALOG: path.join(fixtureRoot, "missing.json"),
126+
}),
127+
).toBeNull();
128+
} finally {
129+
fs.rmSync(fixtureRoot, { force: true, recursive: true });
130+
}
131+
});
132+
133+
it("rejects an embedded catalog without an exact candidate revision (#9464)", () => {
134+
const fixtureRoot = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-live-e2e-catalog-"));
135+
const catalogPath = path.join(fixtureRoot, "catalog.json");
136+
fs.writeFileSync(catalogPath, "{}\n", { mode: 0o600 });
137+
try {
138+
expect(() =>
139+
liveE2eManagedImageCatalog({
140+
GITHUB_ACTIONS: "true",
141+
NEMOCLAW_RUN_LIVE_E2E: "1",
142+
NEMOCLAW_E2E_MANAGED_IMAGE_CATALOG: catalogPath,
143+
}),
144+
).toThrow("requires an exact candidate revision");
145+
} finally {
146+
fs.rmSync(fixtureRoot, { force: true, recursive: true });
147+
}
148+
});
149+
88150
it.each(
89151
SHIPPED_MANAGED_IMAGE_AGENTS,
90152
)("resolves the complete release catalog and exact %s image (#7744)", async (agent) => {
@@ -122,6 +184,23 @@ describe("sandbox workload preparation", () => {
122184
});
123185
});
124186

187+
it("rejects an exact catalog from another PR commit (#9464)", async () => {
188+
const fixtureRoot = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-managed-catalog-"));
189+
const catalogPath = path.join(fixtureRoot, "catalog.json");
190+
fs.writeFileSync(catalogPath, JSON.stringify(CATALOG), { mode: 0o600 });
191+
try {
192+
await expect(
193+
prepareSandboxWorkloadSource({
194+
...input("openclaw"),
195+
catalogPath,
196+
expectedCatalogRevision: "b".repeat(40),
197+
}),
198+
).rejects.toThrow("does not match the live E2E candidate revision");
199+
} finally {
200+
fs.rmSync(fixtureRoot, { force: true, recursive: true });
201+
}
202+
});
203+
125204
it("loads an exact local all-agent catalog without using the registry resolver (#7744)", async () => {
126205
const fixtureRoot = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-managed-catalog-"));
127206
const catalogPath = path.join(fixtureRoot, "catalog.json");

0 commit comments

Comments
 (0)