Skip to content

Commit 74a6e68

Browse files
authored
fix(onboard): retain qualification catalog on rebuild (#9396)
<!-- markdownlint-disable MD041 --> ## Summary Managed-image rebuild preflight now retains the immutable qualification revision used by initial onboarding. Rebuilds outside live GitHub Actions qualification continue to resolve the current release catalog. ## Related Issue Fixes #9385 ## Changes - Resolve the complete managed-image catalog from an exact source revision during live qualification. - Carry the qualification revision through both fresh onboarding and managed rebuild preflight. - Reject malformed revisions and image-label substitutions before image selection. - Preserve current release-catalog rebuild behavior outside GitHub Actions. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Self-review covered public GHCR revision validation, all-agent cohort binding, release-path isolation, and the unchanged credential flow; focused negative tests reject malformed and label-mismatched revisions. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable - Station profile/scenario: Not applicable - Result: Not applicable - Supporting evidence: Not applicable ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Vitest passed 39/39 managed-image catalog tests, 26/26 workload-preparation tests, 3/3 focused rebuild regression tests, 3/3 onboarding-orchestration tests, and 22/22 growth-guardrail tests. `tsc -p tsconfig.cli.json --noEmit` passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Managed workload onboarding and rebuilds now honor immutable managed-image revisions when available. * Invalid, conflicting, or mismatched image revisions are rejected before workload preparation. * Existing behavior remains unchanged outside supported CI environments. * **Tests** * Added coverage for revision validation, onboarding, rebuilds, and catalog resolution scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
1 parent 12e5bee commit 74a6e68

8 files changed

Lines changed: 252 additions & 5 deletions

File tree

src/lib/onboard/managed-image-catalog.test.ts

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,66 @@ describe("managed image GHCR catalog", () => {
411411
},
412412
);
413413

414+
it("resolves an immutable qualification revision as one exact cohort (#9385)", async () => {
415+
const fixture = catalogFixture({ openclaw: { rootReference: REVISION } });
416+
417+
const catalog = await resolveManagedImageCatalogFromGhcr({
418+
release: RELEASE,
419+
revision: REVISION,
420+
fetchImpl: fixture.fetchImpl,
421+
});
422+
423+
expect(
424+
SHIPPED_MANAGED_IMAGE_AGENTS.map(
425+
(agent) =>
426+
(catalog[agent] as { source: { cohort: string; release: string; revision: string } })
427+
.source,
428+
),
429+
).toEqual(
430+
SHIPPED_MANAGED_IMAGE_AGENTS.map(() => ({
431+
cohort: COHORT,
432+
release: RELEASE,
433+
repository: MANAGED_IMAGE_SOURCE_REPOSITORY,
434+
revision: REVISION,
435+
})),
436+
);
437+
const rootManifestRequests = fixture.fetchMock.mock.calls
438+
.map(([input]) => new URL(String(input)).pathname)
439+
.filter((pathname) => pathname.includes("/manifests/"));
440+
expect(rootManifestRequests).toContain(
441+
`/v2/nvidia/nemoclaw/openclaw-sandbox/manifests/${REVISION}`,
442+
);
443+
expect(rootManifestRequests).not.toContain(
444+
`/v2/nvidia/nemoclaw/openclaw-sandbox/manifests/${RELEASE}`,
445+
);
446+
});
447+
448+
it("rejects a malformed qualification revision before registry access (#9385)", async () => {
449+
const fetchImpl = vi.fn();
450+
451+
await expect(
452+
resolveManagedImageCatalogFromGhcr({
453+
release: RELEASE,
454+
revision: "main",
455+
fetchImpl: fetchImpl as typeof fetch,
456+
}),
457+
).rejects.toThrow(/managed image revision 'main' is not a full lowercase SHA/);
458+
expect(fetchImpl).not.toHaveBeenCalled();
459+
});
460+
461+
it("rejects an image that does not match the qualification revision (#9385)", async () => {
462+
const requestedRevision = "c".repeat(40);
463+
const fixture = catalogFixture({ openclaw: { rootReference: requestedRevision } });
464+
465+
await expect(
466+
resolveManagedImageCatalogFromGhcr({
467+
release: RELEASE,
468+
revision: requestedRevision,
469+
fetchImpl: fixture.fetchImpl,
470+
}),
471+
).rejects.toThrow(/source revision does not match the expected revision/);
472+
});
473+
414474
it("fails closed when a dependent cohort alias is torn or absent", async () => {
415475
const fixture = catalogFixture({ hermes: { missingRoot: true } });
416476

@@ -483,7 +543,7 @@ describe("managed image GHCR catalog", () => {
483543
release: RELEASE,
484544
fetchImpl: fixture.fetchImpl,
485545
}),
486-
).rejects.toThrow(/source revision does not match the OpenClaw revision/);
546+
).rejects.toThrow(/source revision does not match the expected revision/);
487547
});
488548

489549
it.each([

src/lib/onboard/managed-image/catalog.ts

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -518,7 +518,7 @@ async function resolveManagedImageContractAtReferenceFromGhcr(options: {
518518
return invalid(`'${agent}' image publication cohort does not match the OpenClaw cohort`);
519519
}
520520
if (options.expectedRevision !== undefined && identity.revision !== options.expectedRevision) {
521-
return invalid(`'${agent}' image source revision does not match the OpenClaw revision`);
521+
return invalid(`'${agent}' image source revision does not match the expected revision`);
522522
}
523523
const image = MANAGED_IMAGE_REPOSITORIES[agent];
524524
return {
@@ -572,19 +572,27 @@ export async function resolveManagedImageContractFromGhcr(options: {
572572

573573
export async function resolveManagedImageCatalogFromGhcr(options: {
574574
readonly release: string;
575+
/** Immutable source revision selected by a live qualification run. */
576+
readonly revision?: string;
575577
readonly platform?: ManagedImagePlatform;
576578
readonly nodeArchitecture?: string;
577579
readonly fetchImpl?: Fetch;
578580
readonly environment?: NodeJS.ProcessEnv;
579581
}): Promise<ManagedImageContractCatalog> {
580582
const release = normalizeManagedImageRelease(options.release);
583+
const revision = options.revision;
584+
if (revision !== undefined && !REVISION_PATTERN.test(revision)) {
585+
return invalid(`managed image revision '${revision}' is not a full lowercase SHA`);
586+
}
581587
const platform = resolveCatalogPlatform(options);
582588
return withRegistryFetch(options.fetchImpl, options.environment, async (fetchImpl) => {
583-
const openclaw = await resolveManagedImageContractFromGhcr({
589+
const openclaw = await resolveManagedImageContractAtReferenceFromGhcr({
584590
agent: "openclaw",
591+
reference: revision ?? release,
585592
release,
586593
platform,
587594
fetchImpl,
595+
...(revision === undefined ? {} : { expectedRevision: revision }),
588596
});
589597
const cohortReference = `cohort-${openclaw.source.cohort}`;
590598
const dependentResults = await Promise.allSettled(

src/lib/onboard/managed-workload/onboard-orchestration.test.ts

Lines changed: 78 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,86 @@
33

44
import { describe, expect, it, vi } from "vitest";
55

6-
import { prepareOnboardSandboxWorkloadLaunch } from "./onboard-orchestration";
6+
const prepareSandboxWorkloadSource = vi.hoisted(() => vi.fn());
7+
8+
vi.mock("../workload/preparation", async (importOriginal) => ({
9+
...(await importOriginal<typeof import("../workload/preparation")>()),
10+
prepareSandboxWorkloadSource,
11+
}));
12+
13+
vi.mock("../../core/version", () => ({ getVersion: () => "v0.0.0" }));
14+
15+
import {
16+
createManagedWorkloadOnboardRuntime,
17+
prepareOnboardSandboxWorkloadLaunch,
18+
} from "./onboard-orchestration";
19+
20+
function createFreshOnboardingRuntime(environment: Readonly<Record<string, string>>) {
21+
const prepared = {
22+
source: {
23+
kind: "legacy-dockerfile",
24+
dockerfilePath: "agents/openclaw/Dockerfile",
25+
reason: "managed-image-unavailable",
26+
},
27+
release: "v0.0.0",
28+
fallbackDiagnostic: null,
29+
};
30+
prepareSandboxWorkloadSource.mockClear();
31+
prepareSandboxWorkloadSource.mockResolvedValueOnce(prepared);
32+
33+
const runtime = createManagedWorkloadOnboardRuntime(
34+
{
35+
computePlan: { driverName: "docker" },
36+
managedWorkloadRebuild: null,
37+
tempManagedRuntime: false,
38+
tempManagedRuntimeCatalog: null,
39+
agentName: "openclaw",
40+
legacyDockerfilePath: "agents/openclaw/Dockerfile",
41+
customDockerfilePath: null,
42+
rootDir: "/tmp/nemoclaw",
43+
model: "model",
44+
provider: "provider",
45+
preferredInferenceApi: null,
46+
endpointUrl: null,
47+
startupProfile: { environment },
48+
note: vi.fn(),
49+
fallbackBuildEstimate: () => null,
50+
} as unknown as Parameters<typeof createManagedWorkloadOnboardRuntime>[0],
51+
{
52+
resolveAgentInferenceApi: vi.fn(),
53+
getSandboxInferenceConfig: vi.fn(),
54+
},
55+
);
56+
57+
return { prepared, runtime };
58+
}
759

860
describe("managed workload onboard orchestration", () => {
61+
it("retains the live qualification catalog revision during fresh onboarding (#9385)", async () => {
62+
const catalogRevision = "a".repeat(40);
63+
const { prepared, runtime } = createFreshOnboardingRuntime({
64+
GITHUB_ACTIONS: "true",
65+
E2E_MANAGED_IMAGE_REVISION: catalogRevision,
66+
});
67+
68+
await expect(runtime.ensurePreparedWorkload()).resolves.toBe(prepared);
69+
expect(prepareSandboxWorkloadSource).toHaveBeenCalledExactlyOnceWith(
70+
expect.objectContaining({ catalogRevision }),
71+
);
72+
});
73+
74+
it("omits the qualification catalog revision outside GitHub Actions (#9385)", async () => {
75+
const { prepared, runtime } = createFreshOnboardingRuntime({
76+
E2E_MANAGED_IMAGE_REVISION: "a".repeat(40),
77+
});
78+
79+
await expect(runtime.ensurePreparedWorkload()).resolves.toBe(prepared);
80+
expect(prepareSandboxWorkloadSource).toHaveBeenCalledOnce();
81+
expect(prepareSandboxWorkloadSource.mock.calls[0]?.[0]).not.toHaveProperty(
82+
"catalogRevision",
83+
);
84+
});
85+
986
it("resolves final-image patch metadata after managed build-context staging", async () => {
1087
const resolutionMetadata = { key: "published-dcode-base" };
1188
let staged = false;

src/lib/onboard/managed-workload/onboard-orchestration.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ import {
5050
import { getSandboxReadyTimeoutSecs } from "../sandbox-gpu-create";
5151
import type { SandboxGpuConfig } from "../sandbox-gpu-mode";
5252
import {
53+
liveE2eManagedImageRevision,
5354
type PreparedSandboxWorkloadSource,
5455
prepareSandboxWorkloadSource,
5556
} from "../workload/preparation";
@@ -155,6 +156,7 @@ export function createManagedWorkloadOnboardRuntime(
155156
let preparedProfile: BuiltManagedStartupOnboardProfile | null = null;
156157

157158
const ensurePreparedWorkload = async (): Promise<PreparedSandboxWorkloadSource> => {
159+
const catalogRevision = liveE2eManagedImageRevision(input.startupProfile.environment);
158160
preparedWorkloadPromise ??= input.managedWorkloadRebuild
159161
? Promise.resolve(
160162
prepareSandboxWorkloadSourceFromRebuildHandoff(
@@ -170,6 +172,7 @@ export function createManagedWorkloadOnboardRuntime(
170172
runtime: runtimeCapabilities,
171173
version: getVersion({ rootDir: input.rootDir }),
172174
catalogPath: input.tempManagedRuntimeCatalog,
175+
...(catalogRevision ? { catalogRevision } : {}),
173176
acceptedCandidateContract: isCandidateAgent(input.agentName)
174177
? readCandidateQualificationReceipt(input.agentName)
175178
: null,

src/lib/onboard/sandbox-workload-preparation.test.ts

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,21 @@ describe("sandbox workload preparation", () => {
107107
});
108108
});
109109

110+
it("passes an immutable qualification revision to catalog resolution (#9385)", async () => {
111+
const resolveCatalog = vi.fn(async () => CATALOG);
112+
113+
await prepareSandboxWorkloadSource(
114+
{ ...input("openclaw"), catalogRevision: REVISION },
115+
{ resolveCatalog },
116+
);
117+
118+
expect(resolveCatalog).toHaveBeenCalledExactlyOnceWith({
119+
release: RELEASE,
120+
platform: MANAGED_IMAGE_PLATFORM,
121+
revision: REVISION,
122+
});
123+
});
124+
110125
it("loads an exact local all-agent catalog without using the registry resolver (#7744)", async () => {
111126
const fixtureRoot = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-managed-catalog-"));
112127
const catalogPath = path.join(fixtureRoot, "catalog.json");

src/lib/onboard/sandbox-workload-rebuild.test.ts

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,6 +309,65 @@ describe("managed workload rebuild preflight", () => {
309309
expect(Object.isFrozen(handoff?.replacement.source.contract.source)).toBe(true);
310310
});
311311

312+
it("retains the live qualification revision during rebuild preflight (#9385)", async () => {
313+
const prepare = vi.fn(async () => replacement("langchain-deepagents-code"));
314+
managedWorkloadRebuildDependencies.prepareSandboxWorkloadSource = prepare;
315+
vi.stubEnv("GITHUB_ACTIONS", "true");
316+
vi.stubEnv("E2E_MANAGED_IMAGE_REVISION", "a".repeat(40));
317+
318+
await prepareManagedWorkloadRebuildHandoff(entry("langchain-deepagents-code"), {
319+
runtime: runtime(),
320+
provider: provider(),
321+
version: "0.0.100",
322+
});
323+
324+
expect(prepare).toHaveBeenCalledExactlyOnceWith({
325+
agentName: "langchain-deepagents-code",
326+
legacyDockerfilePath: "managed-rebuild-must-not-stage-this-dockerfile",
327+
runtime: runtime(),
328+
version: "0.0.100",
329+
policy: "require-managed",
330+
catalogRevision: "a".repeat(40),
331+
});
332+
});
333+
334+
it("rejects a qualification revision that conflicts with durable authority (#9385)", async () => {
335+
const prepare = vi.fn(async () => replacement("langchain-deepagents-code"));
336+
managedWorkloadRebuildDependencies.prepareSandboxWorkloadSource = prepare;
337+
vi.stubEnv("GITHUB_ACTIONS", "true");
338+
vi.stubEnv("E2E_MANAGED_IMAGE_REVISION", "c".repeat(40));
339+
340+
await expect(
341+
prepareManagedWorkloadRebuildHandoff(entry("langchain-deepagents-code"), {
342+
runtime: runtime(),
343+
provider: provider(),
344+
version: "0.0.100",
345+
}),
346+
).rejects.toThrow("live qualification revision does not match the durable workload receipt");
347+
expect(prepare).not.toHaveBeenCalled();
348+
});
349+
350+
it("keeps release-catalog rebuild behavior outside GitHub Actions (#9385)", async () => {
351+
const prepare = vi.fn(async () => replacement("openclaw"));
352+
managedWorkloadRebuildDependencies.prepareSandboxWorkloadSource = prepare;
353+
vi.stubEnv("GITHUB_ACTIONS", "false");
354+
vi.stubEnv("E2E_MANAGED_IMAGE_REVISION", "c".repeat(40));
355+
356+
await prepareManagedWorkloadRebuildHandoff(entry("openclaw"), {
357+
runtime: runtime(),
358+
provider: provider(),
359+
version: "0.0.100",
360+
});
361+
362+
expect(prepare).toHaveBeenCalledExactlyOnceWith({
363+
agentName: "openclaw",
364+
legacyDockerfilePath: "managed-rebuild-must-not-stage-this-dockerfile",
365+
runtime: runtime(),
366+
version: "0.0.100",
367+
policy: "require-managed",
368+
});
369+
});
370+
312371
it.each(AGENTS)("prepares an arm64 replacement handoff for %s", async (agent) => {
313372
managedWorkloadRebuildDependencies.prepareSandboxWorkloadSource = vi.fn(async () =>
314373
replacement(agent, "linux/arm64"),

src/lib/onboard/workload/preparation.ts

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ import {
3131
type ResolveManagedImageCatalog = (options: {
3232
readonly release: string;
3333
readonly platform: ManagedImagePlatform;
34+
readonly revision?: string;
3435
}) => Promise<ManagedImageContractCatalog>;
3536

3637
export interface PrepareSandboxWorkloadSourceInput {
@@ -41,10 +42,17 @@ export interface PrepareSandboxWorkloadSourceInput {
4142
readonly version: string;
4243
readonly policy?: ManagedImageSelectionPolicy;
4344
readonly catalogPath?: string | null;
45+
readonly catalogRevision?: string | null;
4446
/** Contract from the repository-accepted candidate qualification receipt. */
4547
readonly acceptedCandidateContract?: ManagedImageContractV1 | null;
4648
}
4749

50+
export function liveE2eManagedImageRevision(environment: NodeJS.ProcessEnv): string | null {
51+
if (environment.GITHUB_ACTIONS !== "true") return null;
52+
const revision = environment.E2E_MANAGED_IMAGE_REVISION?.trim();
53+
return revision ? revision : null;
54+
}
55+
4856
function readExactManagedImageCatalog(catalogPath: string): ManagedImageContractCatalog {
4957
let descriptor: number | null = null;
5058
try {
@@ -280,7 +288,11 @@ export async function prepareSandboxWorkloadSource(
280288
? readExactManagedImageCatalog(input.catalogPath)
281289
: await (
282290
dependencies.resolveCatalog ?? ((options) => resolveManagedImageCatalogFromGhcr(options))
283-
)({ release, platform });
291+
)({
292+
release,
293+
platform,
294+
...(input.catalogRevision ? { revision: input.catalogRevision } : {}),
295+
});
284296
} catch (error) {
285297
if (!(error instanceof ManagedImageCatalogUnavailableError)) {
286298
throw new SandboxWorkloadPreparationError(

src/lib/onboard/workload/rebuild.ts

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ import {
3636
export type { ManagedWorkloadReceipt } from "./authority";
3737

3838
import {
39+
liveE2eManagedImageRevision,
3940
type PreparedSandboxWorkloadSource,
4041
prepareSandboxWorkloadSource,
4142
SandboxWorkloadPreparationError,
@@ -172,13 +173,25 @@ export async function prepareManagedWorkloadRebuildHandoff(
172173
);
173174
}
174175
} else {
176+
const qualificationRevision = liveE2eManagedImageRevision(process.env);
177+
if (
178+
qualificationRevision !== null &&
179+
qualificationRevision !== authority.receipt.sourceRevision
180+
) {
181+
throw new ManagedWorkloadRebuildError(
182+
"the live qualification revision does not match the durable workload receipt",
183+
);
184+
}
175185
try {
176186
replacement = await managedWorkloadRebuildDependencies.prepareSandboxWorkloadSource({
177187
agentName: authority.agent,
178188
legacyDockerfilePath: "managed-rebuild-must-not-stage-this-dockerfile",
179189
runtime: options.runtime,
180190
version: options.version ?? getVersion(),
181191
policy: "require-managed",
192+
...(qualificationRevision
193+
? { catalogRevision: authority.receipt.sourceRevision }
194+
: {}),
182195
});
183196
} catch (error) {
184197
throw new ManagedWorkloadRebuildError(

0 commit comments

Comments
 (0)