Skip to content

Commit ac74291

Browse files
committed
fix(cli): preserve exact base resolution metadata (#9407)
<!-- markdownlint-disable MD041 --> ## Summary Preserve Deep Agents Code base-resolution metadata when Docker reports a different same-repository `RepoDigest` for an exact platform-digest override. The resolver now retains the caller's immutable platform ref through both digest selection and finalization, so the final sandbox Dockerfile receives the required provenance label. Exact-main failures before this fix selected the correct `linux/amd64` platform digest, completed the lifecycle, then failed final evidence with `Deep Agents Code sandbox image is missing base resolution metadata`: - https://github.com/NVIDIA/NemoClaw/actions/runs/32101603265/job/95604176354 - https://github.com/NVIDIA/NemoClaw/actions/runs/32102057346/job/95604587184 An exact-head qualification of the first revision reproduced the same symptom and exposed the remaining handoff: Docker reported another repository digest, `getRepoDigest()` adopted it, and the finalizer independently normalized to it again. The replacement ref could not supply the exact inspected identity, so metadata was dropped: - https://github.com/NVIDIA/NemoClaw/actions/runs/32105930849/job/95615528882 ## Related Issue Fixes #9386 Follow-up to #9400 and #9392 ## Changes - Preserve an exact same-repository SHA-256 override instead of replacing it with a different digest from Docker's `RepoDigests` list. - Prevent finalization from independently rewriting that exact override. - Treat an exact immutable digest ref plus matching local image identity as valid reuse/rebuild proof when Docker omits or reports another `RepoDigest`. - Keep existing RepoDigest normalization for Dockerfile pins, version tags, and other non-override resolution paths. - Continue rejecting sparse RepoDigest evidence for mutable tags and non-exact refs. - Add regression coverage for the live metadata-loss sequence, omitted `RepoDigests`, malformed JSON, and later exact-ref validation. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: the exception is limited to an exact `imageName@sha256:<64 lowercase hex>` override already accepted by the trusted resolver. Repository equality, digest syntax, local image ID, OS, and architecture remain required. Non-exact refs still require matching `RepoDigests`, and Dockerfile-pin normalization is unchanged. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable; `scripts/prepare-dgx-station-host.sh` is unchanged. - Station profile/scenario: Not applicable. - Result: Not applicable. - Supporting evidence: Not applicable. ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [ ] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable — focused pre-commit, commitlint, lint, formatting, DCO, secret scan, and growth guardrails passed. The reused local dependency tree reports TS2883 portability errors in five unchanged Vitest helper files; GitHub CI is the clean-tree broad gate. - [x] Targeted behavior tests pass for the current change set — 23/23 focused resolver, metadata lifecycle, and validation tests passed with a 30-second per-test ceiling; Oxlint and added-line formatting checks passed. - [ ] Applicable broad gate passed — GitHub exact-head CI and two serial Deep Agents Code live passes pending. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved handling of sandbox base images when repository digest metadata is unavailable or differs. * Exact SHA-256 digest references are now preserved and validated correctly. * Invalid non-digest references continue to be rejected. * Image digest and ID metadata remain accurate for valid pinned references. * **Tests** * Added coverage for missing, malformed, and conflicting repository-digest metadata scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
1 parent 5a3a358 commit ac74291

5 files changed

Lines changed: 143 additions & 6 deletions

File tree

src/lib/sandbox-base-image-platform-digest.test.ts

Lines changed: 61 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -197,13 +197,24 @@ describe("sandbox base-image pinned platform digest resolution", () => {
197197
});
198198
});
199199

200-
it("falls back to the Dockerfile-pinned digest when RepoDigests JSON is malformed", () => {
200+
it("preserves metadata for an exact digest when RepoDigests JSON is malformed (#9386)", () => {
201201
dockerMocks.imageInspect.mockImplementation((ref: string) => ({
202202
status: ref === REF ? 0 : 1,
203203
}));
204204
dockerMocks.imageInspectFormat.mockImplementation((format: string, ref: string) =>
205205
(
206-
new Map([[`{{json .RepoDigests}}\0${REF}`, "{not-json"]]).get(`${format}\0${ref}`) ?? ""
206+
new Map([
207+
[`{{json .RepoDigests}}\0${REF}`, "{not-json"],
208+
[
209+
`{{json .}}\0${REF}`,
210+
JSON.stringify({
211+
Id: IMAGE_ID,
212+
RepoDigests: [],
213+
Os: "linux",
214+
Architecture: "amd64",
215+
}),
216+
],
217+
]).get(`${format}\0${ref}`) ?? ""
207218
).trim(),
208219
);
209220

@@ -218,6 +229,11 @@ describe("sandbox base-image pinned platform digest resolution", () => {
218229
digest: DIGEST,
219230
source: "pinned",
220231
pinnedRemoteRef: REF,
232+
metadata: {
233+
ref: REF,
234+
digest: DIGEST,
235+
imageId: IMAGE_ID,
236+
},
221237
});
222238
expect(traceMocks.add).toHaveBeenCalledWith(
223239
"nemoclaw.sandbox_base_image.repodigest_parse_failed",
@@ -226,6 +242,49 @@ describe("sandbox base-image pinned platform digest resolution", () => {
226242
expect(dockerMocks.build).not.toHaveBeenCalled();
227243
});
228244

245+
it("preserves an exact override when Docker reports a different repository digest (#9386)", () => {
246+
dockerMocks.imageInspect.mockImplementation((ref: string) => ({
247+
status: ref === PLATFORM_REF ? 0 : 1,
248+
}));
249+
dockerMocks.imageInspectFormat.mockImplementation((format: string, ref: string) =>
250+
(
251+
new Map([
252+
[`{{json .RepoDigests}}\0${PLATFORM_REF}`, JSON.stringify([REF])],
253+
[
254+
`{{json .}}\0${PLATFORM_REF}`,
255+
JSON.stringify({
256+
Id: IMAGE_ID,
257+
RepoDigests: [REF],
258+
Os: "linux",
259+
Architecture: "amd64",
260+
}),
261+
],
262+
]).get(`${format}\0${ref}`) ?? ""
263+
).trim(),
264+
);
265+
266+
const resolved = resolveSandboxBaseImage({
267+
...resolutionOptions(),
268+
envVar: "NEMOCLAW_SANDBOX_BASE_IMAGE_REF",
269+
env: {
270+
...resolutionOptions().env,
271+
NEMOCLAW_SANDBOX_BASE_IMAGE_REF: PLATFORM_REF,
272+
},
273+
});
274+
275+
expect(resolved).toMatchObject({
276+
ref: PLATFORM_REF,
277+
digest: PLATFORM_DIGEST,
278+
source: "override",
279+
metadata: {
280+
ref: PLATFORM_REF,
281+
digest: PLATFORM_DIGEST,
282+
imageId: IMAGE_ID,
283+
},
284+
});
285+
expect(dockerMocks.build).not.toHaveBeenCalled();
286+
});
287+
229288
it("rejects a pinned resolution hint from a stale Dockerfile pin", () => {
230289
const options = resolutionOptions();
231290
const stalePin = `${IMAGE_NAME}@sha256:${"c".repeat(64)}`;

src/lib/sandbox-base-image.ts

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ function hasCurrentLocalBuildProvenance(
116116
function getRepoDigest(
117117
imageName: string,
118118
imageRef: string,
119+
preserveExactDigestRef = false,
119120
): { digest: string; ref: string } | null {
120121
const referencesExpectedRepository =
121122
imageRef === imageName ||
@@ -147,6 +148,7 @@ function getRepoDigest(
147148
});
148149
return pinnedDigest;
149150
}
151+
if (preserveExactDigestRef && pinnedDigest) return pinnedDigest;
150152
const repoDigest = Array.isArray(repoDigests)
151153
? repoDigests.find((entry) => String(entry).startsWith(`${imageName}@sha256:`))
152154
: null;
@@ -157,6 +159,7 @@ function getRepoDigest(
157159

158160
type PulledCandidateOptions = {
159161
pinnedRemoteRef?: string;
162+
preserveExactDigestRef?: boolean;
160163
refreshBeforeValidation?: boolean;
161164
refreshIfLocalInvalid?: boolean;
162165
};
@@ -299,7 +302,11 @@ function validatePulledCandidate(
299302
return null;
300303
}
301304

302-
const repoDigest = getRepoDigest(imageName, imageRef);
305+
const repoDigest = getRepoDigest(
306+
imageName,
307+
imageRef,
308+
candidateOptions.preserveExactDigestRef === true,
309+
);
303310
return {
304311
ref: repoDigest?.ref || imageRef,
305312
digest: repoDigest?.digest || null,
@@ -485,6 +492,7 @@ export function resolveSandboxBaseImage(
485492
);
486493
}
487494
const resolved = resolvePulledCandidate(options.imageName, override, "override", options, {
495+
preserveExactDigestRef: true,
488496
refreshBeforeValidation: true,
489497
});
490498
if (resolved?.digest) return finish(resolved);

src/lib/sandbox-base-image/resolution-metadata.test.ts

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,33 @@ describe("sandbox base-image resolution metadata lifecycle", () => {
108108
);
109109
});
110110

111+
it("preserves an exact digest resolution when Docker omits RepoDigests (#9386)", () => {
112+
mocks.dockerImageInspectFormat.mockReturnValue(
113+
JSON.stringify({
114+
Id: inspected.Id,
115+
Os: inspected.Os,
116+
Architecture: inspected.Architecture,
117+
}),
118+
);
119+
120+
expect(createSandboxBaseImageResolutionMetadata(options, KEY, publishedResolution)).toEqual(
121+
metadata,
122+
);
123+
});
124+
125+
it("rejects sparse RepoDigests when the resolved reference is not the exact digest", () => {
126+
mocks.dockerImageInspectFormat.mockReturnValue(
127+
JSON.stringify({ ...inspected, RepoDigests: [] }),
128+
);
129+
130+
expect(
131+
createSandboxBaseImageResolutionMetadata(options, KEY, {
132+
...publishedResolution,
133+
ref: `${IMAGE_NAME}:published`,
134+
}),
135+
).toBeNull();
136+
});
137+
111138
it("finalizes a local fallback with identity metadata and no repository digest (#4680)", () => {
112139
const localResolution: SandboxBaseImageResolution = {
113140
ref: options.localTag,

src/lib/sandbox-base-image/resolution-metadata.ts

Lines changed: 30 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,14 @@ export function inspectLocalImageMetadata(imageRef: string): LocalImageMetadata
2525
}
2626
}
2727

28+
function isExactSameRepositoryDigestRef(
29+
imageName: string,
30+
digest: string,
31+
ref: string,
32+
): boolean {
33+
return /^sha256:[0-9a-f]{64}$/u.test(digest) && ref === `${imageName}@${digest}`;
34+
}
35+
2836
export function validateSandboxBaseImageResolutionMetadata(input: {
2937
metadata: SandboxBaseImageResolutionMetadata;
3038
expectedKey: string;
@@ -67,7 +75,10 @@ export function validateSandboxBaseImageResolutionMetadata(input: {
6775
if (metadata.digest) {
6876
const expectedRepoDigest = `${input.imageName}@${metadata.digest}`;
6977
const repoDigests = Array.isArray(inspected.RepoDigests) ? inspected.RepoDigests : [];
70-
if (!repoDigests.some((entry) => String(entry) === expectedRepoDigest)) {
78+
if (
79+
!isExactSameRepositoryDigestRef(input.imageName, metadata.digest, metadata.ref) &&
80+
!repoDigests.some((entry) => String(entry) === expectedRepoDigest)
81+
) {
7182
return { ok: false, reason: "repo_digest_missing" };
7283
}
7384
}
@@ -89,7 +100,19 @@ export function createSandboxBaseImageResolutionMetadata(
89100
if (resolution.digest) {
90101
const expectedRepoDigest = `${options.imageName}@${resolution.digest}`;
91102
const repoDigests = Array.isArray(inspected?.RepoDigests) ? inspected.RepoDigests : [];
92-
if (!repoDigests.some((entry) => String(entry) === expectedRepoDigest)) return null;
103+
// Docker may omit RepoDigests after resolving an exact platform manifest.
104+
// The resolver's exact same-repository digest ref remains immutable proof.
105+
const exactResolvedReference = isExactSameRepositoryDigestRef(
106+
options.imageName,
107+
resolution.digest,
108+
resolution.ref,
109+
);
110+
if (
111+
!exactResolvedReference &&
112+
!repoDigests.some((entry) => String(entry) === expectedRepoDigest)
113+
) {
114+
return null;
115+
}
93116
}
94117

95118
return {
@@ -115,7 +138,11 @@ export function finalizeSandboxBaseImageResolution(
115138
resolution: SandboxBaseImageResolution,
116139
): SandboxBaseImageResolution {
117140
let locallyProvenResolution = resolution;
118-
if (resolution.digest) {
141+
const preserveExactOverride =
142+
resolution.source === "override" &&
143+
resolution.digest !== null &&
144+
isExactSameRepositoryDigestRef(options.imageName, resolution.digest, resolution.ref);
145+
if (resolution.digest && !preserveExactOverride) {
119146
const inspected = inspectLocalImageMetadata(resolution.ref);
120147
const expectedRepoDigest = `${options.imageName}@${resolution.digest}`;
121148
const matchingRepoDigests = Array.isArray(inspected?.RepoDigests)

src/lib/sandbox-base-image/resolution-validation.test.ts

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,22 @@ describe("sandbox base-image resolution validation", () => {
5858
});
5959
});
6060

61+
it("validates an exact digest reference when Docker reports another repository digest (#9386)", () => {
62+
const digest = `sha256:${"a".repeat(64)}`;
63+
const exactMetadata = {
64+
...metadata,
65+
ref: `${metadata.imageName}@${digest}`,
66+
digest,
67+
};
68+
69+
expect(
70+
validate(exactMetadata, {
71+
...inspected,
72+
RepoDigests: [`${metadata.imageName}@sha256:${"b".repeat(64)}`],
73+
}),
74+
).toEqual({ ok: true });
75+
});
76+
6177
it("validates local fallback images by identity without RepoDigests (#4680)", () => {
6278
expect(
6379
validate(

0 commit comments

Comments
 (0)