Skip to content

Commit 0ac5a7c

Browse files
udsy19cvprekshivyasapurvvkumaria
authored
test(e2e): assert the run-plan cloud check wiring in the support lane (#9493)
## Summary The live registry-targets test asserted that a run plan's `e2eCloudExperimentalChecks` equals `cloudExperimentalChecksForOnboarding(onboarding)` — the same function on the same input that `buildLiveTargetRunPlan` used to produce the field. The comparison could not fail, and the wiring in `test/e2e/live/run-plan.ts` was covered nowhere. This removes the tautology from the live lane and asserts the real wiring in `e2e-support`. ## Related Issue Fixes #9492 ## Changes - `test/e2e/live/registry-targets.test.ts` — drop the self-comparing assertion and the vacuous existence check that followed it (25 of 26 targets carry an empty list, so it proved nothing). - `test/e2e/support/e2e-live-registry-discovery.test.ts` — assert that a target whose onboarding declares cloud experimental checks produces a run plan carrying them, and that one without them does not. Net −3 lines. ## Why the support lane `AGENTS.md` (E2E Selection and Authoring) says deterministic planner, registry and fixture logic belongs in unit, integration, package-contract or `e2e-support` tests rather than a live cloud run. This is deterministic planner logic, so the live lane was paying cloud runtime for an assertion that proved nothing. ## Testing `npx vitest run --project e2e-support test/e2e/support/e2e-live-registry-discovery.test.ts` → 10 passed. Negative control, to show the new assertion is not itself vacuous: replacing the wiring in `run-plan.ts` with `plan.e2eCloudExperimentalChecks = []` makes it fail (`1 failed | 9 passed`). Restored afterwards. The assertion it replaces passes unchanged under that same break. ## Type of change - [x] Test change Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Updated cloud check validation to use the shared checks directory before running plan scripts. * Removed the onboarding-specific experimental cloud check validation. * Added end-to-end coverage confirming that deep-agent cloud onboarding checks are included in generated live target run plans. * Improved validation of cloud onboarding and registry discovery behavior across supported scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai --> ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `no-docs-needed` - Evidence: This test-only change replaces a tautological assertion and changes no user-visible behavior. - Agent: Pi CLI <!-- docs-review-head-sha: 5b3cc33 --> <!-- docs-review-agents-blob-sha: 993bdd8 --> --------- Signed-off-by: Udaya Tejas <udayatejas2004@gmail.com> Signed-off-by: Apurv Kumaria <akumaria@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com> Co-authored-by: Prekshi Vyas <34834085+prekshivyas@users.noreply.github.com> Co-authored-by: Apurv Kumaria <akumaria@nvidia.com>
1 parent 2c74cbb commit 0ac5a7c

2 files changed

Lines changed: 6 additions & 9 deletions

File tree

test/e2e/live/registry-targets.test.ts

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ import {
1414
} from "../fixtures/phases/index.ts";
1515
import { listTargets, requireTargets } from "../registry/registry.ts";
1616
import { liveTargetSupport, liveTargetTestTitle } from "../registry/runtime-support.ts";
17-
import { cloudExperimentalChecksForOnboarding } from "./cloud-experimental-check-list.ts";
1817
import { runE2eCloudExperimentalChecks } from "./cloud-experimental-checks.ts";
1918
import {
2019
captureDcodeBaseImageRuntimeEvidence,
@@ -186,14 +185,6 @@ for (const [targetIndex, target] of listTargets().entries()) {
186185

187186
progress.phase("run target-specific cloud checks");
188187
const checkScripts = runPlan.e2eCloudExperimentalChecks ?? [];
189-
expect(checkScripts).toEqual(
190-
cloudExperimentalChecksForOnboarding(target.environment.onboarding),
191-
);
192-
expect(
193-
checkScripts.every((scriptPath) =>
194-
Object.is(fs.existsSync(path.join(REPO_ROOT, scriptPath)), true),
195-
),
196-
).toBe(true);
197188
expect(fs.existsSync(E2E_CLOUD_EXPERIMENTAL_CHECKS_DIR)).toBe(true);
198189
await runE2eCloudExperimentalChecks(target.id, instance.sandboxName, checkScripts, {
199190
artifacts,

test/e2e/support/e2e-live-registry-discovery.test.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33

44
import { describe, expect, it } from "vitest";
55

6+
import { DEEPAGENTS_CLOUD_EXPERIMENTAL_CHECKS } from "../live/cloud-experimental-check-list.ts";
67
import { buildLiveTargetRunPlan } from "../live/run-plan.ts";
78
import { target } from "../registry/builder.ts";
89
import { liveTargetInventoryEntry } from "../registry/run.ts";
@@ -104,6 +105,11 @@ describe("live target registry discovery support", () => {
104105
suiteIds: registered.suiteIds,
105106
phases: ["environment", "onboarding", "state-validation"],
106107
});
108+
const deepAgents = { ...SUPPORTED_ENVIRONMENT, onboarding: "cloud-langchain-deepagents-code" };
109+
110+
expect(buildLiveTargetRunPlan(syntheticTarget(deepAgents)).e2eCloudExperimentalChecks).toEqual(
111+
DEEPAGENTS_CLOUD_EXPERIMENTAL_CHECKS,
112+
);
107113
});
108114

109115
it("inserts lifecycle execution only when the synthetic target requests it", () => {

0 commit comments

Comments
 (0)