Skip to content

Commit 9670218

Browse files
authored
Merge branch 'main' into fix/9303-deprecated-start-guidance
2 parents 4726b8b + 183a9c8 commit 9670218

52 files changed

Lines changed: 1974 additions & 367 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md

Lines changed: 3 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@ Read the generated `plan.json` and show the maintainer:
125125

126126
For the plan's full `origin/main` SHA, require a completed, successful `Release qualification` check from a pre-tag `.github/workflows/e2e.yaml` run.
127127
The workflow planner derives the required jobs from the workflow's E2E metadata.
128-
By default, the check requires every release-required execution result, including `Publish staging Brev Launchable image`, to succeed.
128+
By default, the check requires every release-required E2E execution result, including `Exact staging Brev Launchable`, to succeed.
129129
A repository administrator may waive one or more release-required E2E execution jobs for a documented release exception.
130130
The waiver requires a comma-separated `release_qualification_waived_jobs` list and a `release_qualification_waiver_reason`.
131131
The reason must begin with an ASCII letter or digit and contain 10-500 characters chosen from ASCII letters, digits, spaces, and `.,:;/_()'-`.
@@ -167,20 +167,6 @@ Before showing the confirmation prompt, present the candidate SHA, workflow URL,
167167
For a waived run, also present the waived jobs, their outcomes, the waiver reason, and both recorded actor identities.
168168
No release-note-only delta exception is currently defined.
169169

170-
After image publication succeeds, present this advisory manual validation:
171-
172-
- State that the image-publication job built and published the candidate image to the staging family used by the [NemoClaw staging Launchable](https://brev.nvidia.com/launchable/deploy/now?launchableID=env-3GdbIjswX4fs3VJ6cYRHr5zoQXo).
173-
- Encourage the maintainer to deploy one instance and hand its Brev environment URL to a Codex session that invokes `nemoclaw-maintainer-validate-launchable`.
174-
- Require the manual validation to compare the deployed concrete image with `launchable-image.json`; do not assume that the mutable family still points to the candidate.
175-
- State that browser-control capability is required for Codex to click and verify the web interface.
176-
- State that a securely supplied inference credential is required to complete hosted and sandbox inference validation. Never ask the maintainer to paste the credential into chat.
177-
- Record the manual result as `complete pass`, `partially blocked`, `failed`, or `not run` when the maintainer provides it.
178-
179-
This manual validation is advisory while the automated Launchable path is blocked by issue #8924.
180-
Its absence, partial result, or failure does not block the signing preflight, confirmation prompt, or release tag.
181-
Do not describe successful image publication as successful Launchable, runtime, or inference validation.
182-
Apply the temporary policy in [Pre-Tag E2E Evidence](../nemoclaw-maintainer-policies/references/release-train.md#temporary-staging-launchable-qualification-policy): NemoClaw maintainers own it while #8924 remains open, the successful exact image-publication job and artifact remain required release evidence under normal Actions retention, and the full automated lane returns only after a checksum-pinned Brev release passes deployment through verified cleanup on trusted `main`.
183-
184170
Run the release script's signing preflight before asking for confirmation:
185171

186172
```bash
@@ -337,11 +323,10 @@ If the Announcement is valid, return its URL with the release artifacts and mark
337323
- Plan generation fails: fix the named precondition, then regenerate the plan.
338324
- Documentation workflow state is incomplete: return to `nemoclaw-maintainer-evening`, then repeat
339325
Step 1 after the documentation PR merges.
340-
- Full-mode E2E waits in the Launchable concurrency queue: keep the run pending until the earlier Launchable image-publication job finishes.
326+
- Full-mode E2E waits in the Launchable concurrency queue: keep the run pending until the earlier Launchable E2E job finishes.
341327
- Full-mode E2E ran for another SHA: reject the run and dispatch full mode for the plan candidate SHA.
342328
- No qualifying `Release qualification` exists: inspect the GitHub result and run pre-tag E2E for the planned SHA only when no qualifying run already exists. Use a job waiver only with explicit repository administrator authorization. Do not release until the release script accepts the canonical check.
343-
- Launchable image publication fails: inspect `launchable-image.json` and the producer run, correct the failure, and rerun the affected work. Do not infer image publication from manual Launchable validation.
344-
- Advisory Launchable validation is blocked or fails: record the exact partial result and continue the release flow. Do not convert the result into a release gate or an automated E2E pass.
329+
- Launchable E2E or cleanup fails: inspect the diagnostic artifacts, correct the failure, and rerun the affected E2E work. Do not infer Launchable success from another workflow result.
345330
- `origin/main` moved after plan generation: regenerate the plan and ask for the new confirmation phrase.
346331
- Remote semver tag already exists: stop; do not retag unless the maintainer explicitly starts protected-tag remediation.
347332
- Signing preflight fails: fix the reported Git signer or signing-key failure. Run the preflight again before requesting confirmation.

.agents/skills/nemoclaw-maintainer-e2e/SKILL.md

Lines changed: 27 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: nemoclaw-maintainer-e2e
3-
description: Dispatches and verifies trusted GitHub Actions E2E for NemoClaw maintainers, including manual PR E2E for the latest PR commit and staging Launchable image publication. Use for requests such as run E2E for PR #123, run the E2E suite, publish the Launchable image, run the Launchable E2E, run the full E2E suite, deploy pre-release full E2E, run pre-tag full E2E, or run release-candidate E2E.
3+
description: Dispatches and verifies trusted GitHub Actions E2E for NemoClaw maintainers, including manual PR E2E for the latest PR commit. Use for requests such as run E2E for PR #123, run the E2E suite, run the Launchable E2E, run the full E2E suite, deploy pre-release full E2E, run pre-tag full E2E, or run release-candidate E2E.
44
---
55

66
<!-- SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -->
@@ -34,14 +34,17 @@ The workflow does not rotate or revoke these API keys or messaging credentials.
3434
Live targets can create external resources.
3535
After a failure, inspect the artifacts and remove resources that target cleanup did not remove.
3636

37-
`Publish staging Brev Launchable image` reads this credential from repository Actions secrets:
37+
`Exact staging Brev Launchable` reads these credentials from repository Actions secrets:
3838

39+
- `BREV_API_KEY` authenticates the trusted host-side Brev CLI for workspace operations in the organization identified by `BREV_ORG_ID`. Candidate code does not receive this API key.
3940
- `NEMOCLAW_IMAGE_DISPATCH_TOKEN` is exposed as `GH_TOKEN` only to the trusted host script. It grants Actions read/write access to `brevdev/nemoclaw-image`, which the script uses to dispatch the image workflow, inspect its run, and download its handoff artifact.
41+
- `NVIDIA_INFERENCE_API_KEY` is exported into the Brev guest for the full E2E process. Code in the baked candidate checkout can read and use it.
4042

41-
This credential remains valid until it expires or an administrator revokes it in GitHub. Rotate or revoke it to remove later access.
42-
The job does not receive `BREV_API_KEY`, `BREV_ORG_ID`, or `NVIDIA_INFERENCE_API_KEY`.
43-
It does not install or authenticate the Brev CLI, create a workspace, or run inference.
44-
This image-publication credential boundary applies only to trusted Launchable or full manual dispatches against `main`. It does not apply to `main` pushes or manual PR runs.
43+
`brev login` writes `BREV_API_KEY` and `BREV_ORG_ID` to `$HOME/.brev/credentials.json` on the GitHub-hosted runner. Later trusted steps and processes in the same job can read that file. The workflow does not delete it explicitly; it remains on the ephemeral runner filesystem until runner teardown discards that filesystem.
44+
These credentials remain valid until they expire or an administrator revokes them in their issuing services. If cleanup fails, remove the recorded Brev workspace. Rotate or revoke each credential to remove later access.
45+
This Brev credential boundary applies only to trusted Launchable or full manual dispatches against `main`. It does not apply to `main` pushes or manual PR runs.
46+
47+
The `NEMOCLAW_STAGING_LAUNCHABLE_ID` repository Actions variable selects the standing Launchable. Keep its value equal to the Launchable ID in the default URL owned by [`nemoclaw-maintainer-validate-launchable`](../nemoclaw-maintainer-validate-launchable/SKILL.md).
4548

4649
For `managed-image-protected-runtime`, the workflow supplies the long-lived `NVIDIA_API_KEY` repository secret only to the trusted qualification step. Trusted host code uses it for NGC login and passes it as `NGC_API_KEY` and `NIM_NGC_API_KEY` to the temporary NIM container. Candidate managed sandboxes receive generated local route tokens instead of this key. The live fixture removes the temporary NIM container only if its exact ID, name, requested image, immutable image ID, cohort owner, and provider kind match the recorded authority. The test fails if evidence is missing or ambiguous, a name is reused, authority drifts, removal is indeterminate, or the exact ID or name remains. A cleanup refusal can leave the container and its API key in place until runner teardown. The final workflow step removes the job's isolated Docker credential directory and fails if that removal does not complete. The workflow does not revoke the NVIDIA API key. Revoke it, or rotate it and disable the old value, in the issuing NVIDIA service. Verify that the exposed key is no longer valid.
4750

@@ -67,7 +70,7 @@ Require a review reason containing 10 to 500 printable characters.
6770
Choose exactly one mode:
6871

6972
- For a PR revision run, leave `E2E_JOBS` empty. The run selects:
70-
- every default-selected free-standing workflow E2E except `Publish staging Brev Launchable image`;
73+
- every default-selected free-standing workflow E2E except `Exact staging Brev Launchable`;
7174
- every shared credential-free test; and
7275
- these controller-selected registry targets: `ubuntu-policy-custom-missing-presets-negative`, `ubuntu-repo-cloud-langchain-deepagents-code`, `ubuntu-repo-cloud-openclaw`, and `ubuntu-repo-docker-post-reboot-recovery`.
7376
The run skips `jetson-nvmap-gpu` unless `allow_jetson_dispatch` is `true`.
@@ -158,25 +161,20 @@ A changed head repository, head SHA, or base SHA invalidates the evidence and re
158161
| Request | Mode | `jobs` | `include_staging_brev_launchable` |
159162
|---|---|---|---|
160163
| “Run the E2E suite” | Ordinary | empty | `false` |
161-
| “Publish the Launchable image” | Launchable image | `staging-brev-launchable` | `false` |
162-
| “Run the Launchable E2E” | Clarify before dispatch | not applicable | not applicable |
164+
| “Run the Launchable E2E” | Launchable | `staging-brev-launchable` | `false` |
163165
| “Run the full E2E suite” | Full | empty | `true` |
164166
| “deploy pre-release full E2E” | Full | empty | `true` |
165167
| “run pre-tag full E2E” | Full | empty | `true` |
166168
| “run release-candidate E2E” | Full | empty | `true` |
167169
| “run pre-tag E2E with an administrator job waiver” | Administrator-waived full | empty | `true` |
168170
169171
A generic E2E request must not authorize the Brev Launchable path.
170-
For “Run the Launchable E2E,” explain that issue #8924 blocks automated deployment, runtime, and inference validation.
171-
Ask whether the maintainer wants image publication or advisory validation through `nemoclaw-maintainer-validate-launchable` against one deployed instance.
172-
Do not dispatch until the maintainer selects one of those operations.
173172
Do not infer full mode from words such as “all” or “complete.”
174-
Ask for clarification when the request uses the legacy Launchable E2E phrase or contains conflicting mode phrases.
173+
Ask for clarification only when the request contains conflicting mode phrases.
175174
176-
Ordinary mode selects every default-selected workflow E2E except `Publish staging Brev Launchable image`.
177-
Launchable image mode runs only `Publish staging Brev Launchable image`.
178-
Full mode adds `Publish staging Brev Launchable image` to the default E2E selection in the same workflow run.
179-
The Launchable image job stops after exact image-publication evidence and does not deploy a workspace or run inference.
175+
Ordinary mode selects every default-selected workflow E2E except `Exact staging Brev Launchable`.
176+
Launchable mode runs only `Exact staging Brev Launchable`.
177+
Full mode adds `Exact staging Brev Launchable` to the default E2E selection in the same workflow run.
180178
Administrator-waived full mode runs the full suite but omits the approved execution jobs from release qualification.
181179
Every waived job still runs.
182180
Use this mode only when a repository administrator explicitly authorizes the job IDs and supplies the reason.
@@ -222,7 +220,7 @@ gh workflow run .github/workflows/e2e.yaml \
222220
-f "correlation_id=${CORRELATION_ID}"
223221
```
224222
225-
For Launchable image mode:
223+
For Launchable mode:
226224
227225
```bash
228226
gh workflow run .github/workflows/e2e.yaml \
@@ -273,8 +271,8 @@ gh workflow run .github/workflows/e2e.yaml \
273271
```
274272
275273
Do not set `jobs=staging-brev-launchable` for full mode.
276-
Empty `jobs` and `targets` select every default-selected workflow E2E except `Publish staging Brev Launchable image`.
277-
The `include_staging_brev_launchable` input adds the Launchable image-publication job to that same run.
274+
Empty `jobs` and `targets` select every default-selected workflow E2E except `Exact staging Brev Launchable`.
275+
The `include_staging_brev_launchable` input adds the Launchable E2E job to that same run.
278276
The trusted `main` workflow verifies that the dispatching and rerunning actors have
279277
repository `maintain` or `admin` permission before the Launchable path's source
280278
checkout. That role check is the authorization.
@@ -300,7 +298,7 @@ empty-selector manual run or enable explicit qualification selection. Set it
300298
only after a repository administrator confirms an online DGX Spark runner in
301299
the authoritative runner inventory.
302300
If GitHub pauses the qualification job for the `approve-dgx-spark-image-qualification` environment, an authorized environment reviewer must approve it before qualification starts.
303-
`Publish staging Brev Launchable image` does not require environment approval.
301+
`Exact staging Brev Launchable` does not require environment approval.
304302
305303
Find the run by its unique title:
306304
@@ -334,7 +332,7 @@ Wait for completion:
334332
gh run watch "$RUN_ID" --repo NVIDIA/NemoClaw
335333
```
336334
337-
Launchable image and full modes can wait in the non-cancelling Launchable concurrency queue.
335+
Launchable and full modes can wait in the non-cancelling Launchable concurrency queue.
338336
Queued, waiting, or accepted dispatch state is not success.
339337
Classify the completed workflow and `Release qualification` job with the checks below.
340338
@@ -356,21 +354,20 @@ Require `run-$RUN_ID.json` to report:
356354
- `head_sha` equal to `CANDIDATE_SHA`;
357355
- `status` equal to `completed`.
358356
359-
For ordinary, Launchable image, and unwaived full modes, require `conclusion` equal to `success`.
357+
For ordinary, Launchable, and unwaived full modes, require `conclusion` equal to `success`.
360358
For administrator-waived full mode, permit `conclusion` equal to `success` or `failure`.
361359
A `failure` conclusion is acceptable only when one completed, successful `Release qualification` job and a valid exact-run waiver artifact with at least one canonical waived job failure both exist.
362360
363-
For Launchable image mode, also require `jobs-latest-$RUN_ID.json` to contain one completed, successful
364-
`Publish staging Brev Launchable image` job. Return the workflow and job URLs.
365-
Require its artifact to contain `launchable-image.json` for the selected candidate SHA and concrete staging image URI.
361+
For Launchable mode, also require `jobs-latest-$RUN_ID.json` to contain one completed, successful
362+
`Exact staging Brev Launchable` job. Return the workflow and job URLs.
366363
367364
For a full run, with or without a job waiver, require `jobs-latest-$RUN_ID.json` to contain one completed, successful
368365
`Release qualification` job. Return its job URL with the workflow URL.
369-
In full mode, that job waits for every default-required result, including `Publish staging Brev Launchable image`.
370-
The Launchable image job verifies only the exact candidate image producer receipt and staging-family publication.
371-
Its `launchable-image.json` artifact records Launchable, runtime, and inference validation as not run.
366+
In full mode, that job waits for every default-required E2E result, including `Exact staging Brev Launchable`.
367+
The Launchable job directly verifies the candidate checkout, in-guest full E2E result, and workspace cleanup before it succeeds.
368+
Its `launchable-e2e.json`, `full-e2e.log`, and `cleanup.json` artifacts remain available for diagnosis.
372369
A skipped, cancelled, queued, or failed `Release qualification` job is not evidence.
373-
A Launchable image-only run is not full-mode or pre-tag release evidence.
370+
A Launchable-only run is not full-mode or pre-tag release evidence.
374371
375372
For administrator-waived full mode, the job waits for every unwaived release-required result.
376373
A waived execution job may fail without failing `Release qualification`.

0 commit comments

Comments
 (0)