test(e2e): restore reviewed legacy gateway fixture - #9398
Merged
Conversation
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Included review availability: Your plan includes up to 12 reviews per rolling hour; 0 remain after this review. 📝 WalkthroughWalkthroughThe legacy keepalive fixture now validates managed Docker inspect output, rewrites approved containers to the legacy supervisor contract, injects Docker capture during recreation, and tests valid, invalid, identity-mismatch, and GPU startup-command cases. ChangesLegacy keepalive fixture
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to The change is localized to the legacy gateway recovery fixture and its tests; no actionable merge-blocking risk remains after normal checks and review. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in commit 9012adf in the TypeScript / code-coverage/cliThe overall coverage in commit 9012adf in the Show a code coverage summary of the most impacted files.
Updated |
Contributor
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
3 terminology differences from the second opinionAdvisory only. These are normalized differences from the primary terminology receipt.
2 additional E2E selections from the second opinionAdvisory only. The primary lane did not select these E2E jobs or targets.
Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests. 3 semantic terminology decisionsTerminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.
E2E guidanceAdvisory only. A maintainer can dispatch the default E2E suite for the commit under review. Recommended E2E: None Manual-only E2E: 1 optional E2E recommendation
This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
prekshivyas
approved these changes
Aug 18, 2026
prekshivyas
left a comment
prekshivyas
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed exact commit d352abb2b4302755db49f3cc08ad1eac98695d55.
Security review:
- Secrets and credentials — PASS: the fixture adds no credential material.
- Input validation — PASS: rewriting requires the exact container identity, entrypoint, and command.
- Authentication and authorization — PASS: production authorization is unchanged.
- Dependencies and supply chain — PASS: no dependency changes.
- Error handling and logging — PASS: mismatched fixture state is rejected.
- Cryptography — PASS: no cryptographic changes.
- Policy and network isolation — PASS: the production clone validator remains unchanged.
- Testing and regression safety — PASS: identity, entrypoint, accepted command, and rejected command paths are covered; focused fixture assertions passed locally.
- System security — PASS: the compatibility rewrite is limited to the injected legacy E2E fixture.
Full CI is still a separate merge gate.
20 tasks
prekshivyas
pushed a commit
that referenced
this pull request
Aug 18, 2026
<!-- markdownlint-disable MD041 --> ## Summary Legacy gateway recovery now repeats the OpenShell readiness check and managed supervisor health check after the final replacement container restart. NemoClaw starts the primary dashboard/API host forward only after both checks pass. Recovery stops if Docker cannot stop or start the replacement container, or if either check fails. ### E2E claim - Root-cause key: `gateway recovery / legacy supervisor relaunch final handoff / post-finalize OpenShell readiness not re-established before primary forward start` - Source workflow: [run `32089878527`](https://github.com/NVIDIA/NemoClaw/actions/runs/32089878527), attempt `1` - Failed job: `Gateway: restores the guard chain after recreation / NVIDIA inference API key` ([job `95570032040`](https://github.com/NVIDIA/NemoClaw/actions/runs/32089878527/job/95570032040)) - Failure signature: `Probe failed: OpenClaw gateway is running in 'e2e-2701', but the primary dashboard/API host forward could not be re-established.` - Claim scope: legacy managed-supervisor recovery after the final replacement container restart. ## Related Issue Follow-up to #9364. Issue #9364 defines the legacy compatibility acceptance criterion. PR #9398 corrected its legacy fixture, which let the E2E test reach this later recovery failure. This change preserves the production supervisor allowlist. ## Changes - Run the shared final Docker handoff function in the legacy `sleep infinity` recovery regression test. - Repeat the pinned managed supervisor health check after Docker restarts the replacement container. - Repeat the OpenShell readiness check before NemoClaw starts the primary dashboard/API host forward. - Stop recovery and report the failed layer when final Docker or readiness verification fails. - Keep prerequisite recovery failures separate from forward failures and redact their diagnostics. - Preserve the production supervisor allowlist. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent security rubric review of commit under review `75c124bb66eadd9706da7fcef13581f93e0773e2` returned PASS with no findings. Live E2E for this commit remains required. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable - Station profile/scenario: Not applicable - Result: Not applicable - Supporting evidence: Not applicable ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npm exec -- vitest run --project integration test/process-recovery-supervisor-relaunch.test.ts` (35 passed); `npm exec -- vitest run --project cli src/lib/actions/sandbox/connect-flow.test.ts` (40 passed); `npm exec -- vitest run --project e2e-support test/e2e/support/launch-agent-turn.test.ts` (19 passed, 24 platform-skipped); `npm run typecheck:cli` passed; `npm run checks:repository` passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable to this focused recovery change; PR CI will run for latest PR commit `75c124bb66eadd9706da7fcef13581f93e0773e2`. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Documentation impact: No public documentation change. Existing recovery documentation already describes the corrected contract. --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved sandbox recovery failure reporting with clearer, sanitized details. - Prevented irrelevant manual-forward guidance when recovery fails before forwarding. - Preserved the most direct recovery error when multiple failure layers are reported. - Improved supervisor relaunch recovery with readiness checks, rollback handling, container handoff validation, and cleanup warnings. - Correctly reports container identity changes and managed health failures as recovery failures. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
ericksoa
added a commit
that referenced
this pull request
Aug 18, 2026
<!-- markdownlint-disable MD041 --> ## Summary The gateway guard recovery E2E previously modeled a legacy runtime by replacing the managed image entrypoint with raw `sleep infinity`, which the production supervisor allowlist correctly rejects. This change makes the fixture use the reviewed pre-0.0.99 OpenShell supervisor contract while leaving the production guard unchanged. ## Related Issue Fixes #9364 ## Changes - Adapt the identity-pinned managed container inspect record only when it has the exact managed-image entrypoint and command, then recreate it under the reviewed pre-0.0.99 OpenShell supervisor contract for the legacy recovery fixture. - Keep `docker-gpu-patch-clone.ts` unchanged so arbitrary entrypoints, commands, and supervisor shapes remain rejected. A direct production change would weaken the contract that this E2E is intended to protect. - Add fixture tests for the accepted legacy contract, the exact source container identity and managed process contract, and compatibility with the production clone validator. The existing production denial suite continues to protect rejection behavior. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Exact-head PR Review Advisor run 32095060441 passed both lanes with `merge_as_is`; CodeRabbit completed with no actionable comments. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable - Station profile/scenario: Not applicable - Result: Not applicable - Supporting evidence: Not applicable ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npx vitest run --project e2e-support --project cli test/e2e/support/gateway-guard-legacy-keepalive-fixture.test.ts src/lib/onboard/docker-gpu-patch-validation.test.ts` (28 passed); `npm run test:e2e-phases:check` (131 tests across 86 files). - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable; this is a narrow E2E fixture correction with focused behavior and production-denial coverage. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> --------- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com>
ericksoa
pushed a commit
that referenced
this pull request
Aug 18, 2026
<!-- markdownlint-disable MD041 --> Legacy gateway recovery now repeats the OpenShell readiness check and managed supervisor health check after the final replacement container restart. NemoClaw starts the primary dashboard/API host forward only after both checks pass. Recovery stops if Docker cannot stop or start the replacement container, or if either check fails. - Root-cause key: `gateway recovery / legacy supervisor relaunch final handoff / post-finalize OpenShell readiness not re-established before primary forward start` - Source workflow: [run `32089878527`](https://github.com/NVIDIA/NemoClaw/actions/runs/32089878527), attempt `1` - Failed job: `Gateway: restores the guard chain after recreation / NVIDIA inference API key` ([job `95570032040`](https://github.com/NVIDIA/NemoClaw/actions/runs/32089878527/job/95570032040)) - Failure signature: `Probe failed: OpenClaw gateway is running in 'e2e-2701', but the primary dashboard/API host forward could not be re-established.` - Claim scope: legacy managed-supervisor recovery after the final replacement container restart. Follow-up to #9364. Issue #9364 defines the legacy compatibility acceptance criterion. PR #9398 corrected its legacy fixture, which let the E2E test reach this later recovery failure. This change preserves the production supervisor allowlist. - Run the shared final Docker handoff function in the legacy `sleep infinity` recovery regression test. - Repeat the pinned managed supervisor health check after Docker restarts the replacement container. - Repeat the OpenShell readiness check before NemoClaw starts the primary dashboard/API host forward. - Stop recovery and report the failed layer when final Docker or readiness verification fails. - Keep prerequisite recovery failures separate from forward failures and redact their diagnostics. - Preserve the production supervisor allowlist. - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent security rubric review of commit under review `75c124bb66eadd9706da7fcef13581f93e0773e2` returned PASS with no findings. Live E2E for this commit remains required. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: - [ ] Tested on DGX Station - Tested commit: Not applicable - Station profile/scenario: Not applicable - Result: Not applicable - Supporting evidence: Not applicable - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npm exec -- vitest run --project integration test/process-recovery-supervisor-relaunch.test.ts` (35 passed); `npm exec -- vitest run --project cli src/lib/actions/sandbox/connect-flow.test.ts` (40 passed); `npm exec -- vitest run --project e2e-support test/e2e/support/launch-agent-turn.test.ts` (19 passed, 24 platform-skipped); `npm run typecheck:cli` passed; `npm run checks:repository` passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable to this focused recovery change; PR CI will run for latest PR commit `75c124bb66eadd9706da7fcef13581f93e0773e2`. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Documentation impact: No public documentation change. Existing recovery documentation already describes the corrected contract. --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> - **Bug Fixes** - Improved sandbox recovery failure reporting with clearer, sanitized details. - Prevented irrelevant manual-forward guidance when recovery fails before forwarding. - Preserved the most direct recovery error when multiple failure layers are reported. - Improved supervisor relaunch recovery with readiness checks, rollback handling, container handoff validation, and cleanup warnings. - Correctly reports container identity changes and managed health failures as recovery failures. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The gateway guard recovery E2E previously modeled a legacy runtime by replacing the managed image entrypoint with raw
sleep infinity, which the production supervisor allowlist correctly rejects. This change makes the fixture use the reviewed pre-0.0.99 OpenShell supervisor contract while leaving the production guard unchanged.Related Issue
Fixes #9364
Changes
docker-gpu-patch-clone.tsunchanged so arbitrary entrypoints, commands, and supervisor shapes remain rejected. A direct production change would weaken the contract that this E2E is intended to protect.Type of Change
Quality Gates
merge_as_is; CodeRabbit completed with no actionable comments.DGX Station Hardware Evidence
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run validate:prpassed after refreshingorigin/mainwhen hooks were skipped or unavailablenpx vitest run --project e2e-support --project cli test/e2e/support/gateway-guard-legacy-keepalive-fixture.test.ts src/lib/onboard/docker-gpu-patch-validation.test.ts(28 passed);npm run test:e2e-phases:check(131 tests across 86 files).npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result: Not applicable; this is a narrow E2E fixture correction with focused behavior and production-denial coverage.npm run docsbuilds without warnings (doc changes only)Signed-off-by: Aaron Erickson aerickson@nvidia.com