test(integration): drive the integration specs against the live stack (#717)

Author: 0xisk

CONTRIBUTING.md

@@ -26,6 +26,7 @@ We really appreciate and value contributions to OpenZeppelin Contracts for Compa
 [Running Tests](#running-tests)
 
 * [Unit Tests](#unit-tests)
+* [Integration Tests](#integration-tests)
 * [Live Tests](#live-tests)
 
 [Styleguides](#styleguides)
@@ -169,6 +170,14 @@ Unit tests run against an in-process mock backend (no network, ZK proving skippe
 yarn test
 ```
 
+### Integration Tests
+
+Composed-contract specs (`contracts/test/integration/specs`): several modules assembled into one contract under `test/integration/_mocks`, then deployed and driven as a unit. Same mock backend as the unit tests.
+
+```bash
+yarn test:integration
+```
+
 ### Live Tests
 
 Live tests run against a local Midnight network (node, indexer, and proof server) defined in [`local-env.yml`](./local-env.yml). They require [Docker](https://docs.docker.com/get-docker/) and a completed `yarn install`.
@@ -179,29 +188,37 @@ One command runs everything — it compiles, resets the stack, runs a quick harn
 yarn test:live
 ```
 
-Currently `multisig` is the only live-ready category; the others still assume dry-only semantics and are skipped (listed in the run banner). Each category joins the run — with its own `test:live:<category>` script — as its specs are refactored for the live backend.
+Currently `multisig` is the only live-ready category. The others still assume dry-only semantics and are skipped, and the run banner lists them. Each joins the run as its specs are refactored for the live backend, with its own `test:live:<category>` script.
+
+`integration` is a target of its own, not a category, so an unscoped run skips it. Ask for it by name. Only one live target runs per invocation, since both live projects draw wallets from the same genesis-funded pool.
+
+```bash
+yarn test:live integration   # or: yarn test:integration:live
+yarn test:live --list        # the live targets, as the CI matrix reads them
+```
+
+> **Note:** the `integration` live target is the harness capability plus a boundary check, not functional coverage. As of ledger v8 the composed contract does not deploy: its circuits' IR overruns the per-tx block byte budget, and the spec asserts that rejection rather than skipping. A ledger bump can move the budget, so a red spec there means the deploy now fits and the functional specs are worth porting to live.
 
 If any files fail, a second round re-runs just those files on a fresh node with one worker, to separate a real failure from an environment flake:
 
 * Fails round 1, passes round 2 → **FLAKY** (exit 0, reported loudly).
 * Fails both rounds → **REAL** (exit non-zero).
 
-Scope the same mechanism to one category, or to a subset within it. The first
-argument names the category; any further argument is a filename substring
-(vitest matches it), so pass a spec name to run every file whose name matches it
-on the live backend — the fast loop while iterating on one feature, instead of
-waiting for the whole category. The match is a substring, not an exact file, so
-a name that prefixes others runs all of them:
+Scope the same mechanism to one target, or a subset within it. The first argument
+names the target (a category, or `integration`). Any further argument is a
+filename substring vitest matches, which is the fast loop while iterating on one
+feature. Being a substring, a name that prefixes others runs all of them:
 
 ```bash
 yarn test:live multisig                  # the whole category
 yarn test:live multisig ShieldedTreasury # any file matching "ShieldedTreasury"
+yarn test:live integration ConfidentialFungibleToken # one integration spec
 ```
 
 The two-round flake check still applies to a scoped run, so a green result
 means the same thing it does for the full suite.
 
-Stop the network when done: `yarn env:down`. (No manual `env:up` is needed — the runner resets the stack itself.)
+The runner owns the stack: it starts it (`make env-up`, itself a reset) and stops it on every exit path, Ctrl-C included. No manual `env:up` or `env:down` needed. To inspect a run afterwards, set `MIDNIGHT_LIVE_KEEP_ENV=1` and stop it yourself. Container logs land in `logs/` either way.
 
 > **Note:** The live tests all run against one shared node, so state left by an earlier run can make a later one fail. Two rules keep them reliable, both enforced by a guard that fails fast, before any wallet build:
 >
@@ -212,12 +229,13 @@ Environment knobs:
 
 | Variable | Default | Effect |
 | --- | --- | --- |
-| `MIDNIGHT_LIVE_WORKERS` | 3 | Parallel spec files (max 3 — one genesis-funded deployer each). |
+| `MIDNIGHT_LIVE_WORKERS` | 3 | Parallel spec files under `unit-live` (max 3, one genesis-funded deployer each). |
 | `MIDNIGHT_LIVE_ALLOW_DIRTY` | unset | `1` skips the freshness check (run against a dirty node). |
 | `MIDNIGHT_LIVE_MAX_COIN_EVENTS` | 0 | Coin events beyond genesis tolerated before "not fresh". |
 | `MIDNIGHT_LIVE_MAX_SCAN_BLOCKS` | 3600 | Above this indexer head, the guard asks you to `env:up` rather than scan. |
+| `MIDNIGHT_LIVE_KEEP_ENV` | unset | `1` leaves the stack running after the run instead of tearing it down. |
 
-`unit-live` runs up to 3 workers in parallel, so their output interleaves. It is tagged per worker: a `▶ live worker N/3 ready` banner when a worker's wallets are funded, a `[wN] ❯ <file>` line as each spec file starts, and a `[wN] ✓ <test> (<ms>) [done/total]` line per test — showing the worker, the result, and overall progress through the run. Each worker also writes a detailed log to `logs/live-harness-wN.log`.
+`integration-live` runs one worker (only the deployer wallet is in play). `unit-live` runs up to 3 workers in parallel, so their output interleaves. It is tagged per worker: a `▶ live worker N/3 ready` banner when a worker's wallets are funded, a `[wN] ❯ <file>` line as each spec file starts, and a `[wN] ✓ <test> (<ms>) [done/total]` line per test — showing the worker, the result, and overall progress through the run. Each worker also writes a detailed log to `logs/live-harness-wN.log`.
 
 > **Tip:** to save the run to a colored, readable log, force color and pipe to `tee`. Piping (stdout is no longer a TTY) makes vitest print one clean line per result instead of an animated spinner, and `FORCE_COLOR=1` keeps the color. Write it to a `.ansi` file:
 >

contracts/package.json

@@ -32,7 +32,7 @@
     "compile:security": "compact-compiler --dir security",
     "compile:token": "compact-compiler --dir token",
     "compile:utils": "compact-compiler --dir utils",
-    "compile:integration": "SKIP_ZK=true compact-compiler --src test/integration/_mocks",
+    "compile:integration": "compact-compiler --src test/integration/_mocks",
     "build": "compact-builder --hierarchical --out dist --clean-dist --exclude '*/archive/*' --exclude 'Mock*' --exclude '*.mock.compact' --copy package.json --copy ../README.md && find dist -type d -empty -delete",
     "test": "vitest run --project unit",
     "test:coverage": "vitest run --project unit --coverage",

contracts/test-utils/harness/live.globalSetup.ts

@@ -9,7 +9,11 @@ import { fetchCoinEvents, indexerHead } from './ledgerEvents.js';
  * of after a slow wallet build. Gated on `MIDNIGHT_BACKEND === 'live'` so a dry
  * `vitest run` that happens to glob the live tests is a no-op.
  *
- * It guards two things:
+ * It guards three things:
+ *   - **One live project per invocation.** Every live project derives its
+ *     wallets from the same `VITEST_POOL_ID` partition, so two of them in one
+ *     vitest run would spend the same genesis deployer's coins. See
+ *     {@link assertSoleLiveProject}.
  *   - **Freshness.** The live tests are not isolated from one another: they all
  *     run against the same node, so shielded-coin state left by an earlier run
  *     changes a later run's outcome (a coin re-spent against stale state is
@@ -49,6 +53,50 @@ const ENV_UP_HINT = "run 'yarn env:up' to reset the local stack";
 const sleep = (ms: number): Promise<void> =>
   new Promise((resolve) => setTimeout(resolve, ms));
 
+// --- one live project per invocation ---------------------------------------
+
+// Parked on `globalThis`, not in a module-level binding: each project loads this
+// file through its OWN module runner, so a module variable would not be shared
+// between two projects in the same process.
+const PROJECT_CLAIM = '__midnightLiveProject';
+
+/**
+ * Reject a second live project in the same vitest invocation.
+ *
+ * Every live project builds its wallets from `walletSeedsFor(VITEST_POOL_ID)`,
+ * so worker 1 of `unit-live` and worker 1 of `integration-live` resolve to the
+ * SAME genesis deployer seed. Two pools would then balance transactions against
+ * one deployer's UTXO snapshot — the stale-UTXO race (node `Custom error: 103`)
+ * that `WalletPool.ensureReady`'s serial build exists to prevent — and both
+ * would write `logs/live-harness-w1.log`, interleaving two runs' diagnostics.
+ *
+ * The run lock below cannot catch this: it is deliberately reentrant for our own
+ * pid, precisely so one process CAN run several live globalSetups. So the claim
+ * is tracked separately here.
+ *
+ * @param project - the project whose globalSetup is running
+ * @param claimed - the project that already claimed this process, if any
+ */
+export function assertSoleLiveProject(
+  project: string,
+  claimed: string | undefined,
+): void {
+  if (claimed === undefined || claimed === project) return;
+  throw new Error(
+    `two live projects in one vitest run ('${claimed}' and '${project}'): ` +
+      'both derive their wallets from the same VITEST_POOL_ID partition, so ' +
+      "each project's worker 1 would spend the same genesis deployer's coins " +
+      '(node "Custom error: 103"). Pass one --project per invocation.',
+  );
+}
+
+/** Claim this process for `project`, or throw if another live project holds it. */
+function claimLiveProject(project: string): void {
+  const registry = globalThis as Record<string, unknown>;
+  assertSoleLiveProject(project, registry[PROJECT_CLAIM] as string | undefined);
+  registry[PROJECT_CLAIM] = project;
+}
+
 // --- lock ------------------------------------------------------------------
 
 export interface LockInfo {
@@ -206,8 +254,15 @@ async function assertFreshNode(): Promise<void> {
   }
 }
 
-export default async function setup(): Promise<() => void> {
+/**
+ * Vitest calls this once per project, in the main process, passing that project.
+ * Typed structurally so this file keeps importing nothing but `node:` builtins.
+ */
+export default async function setup(project?: {
+  readonly name?: string;
+}): Promise<() => void> {
   if (process.env.MIDNIGHT_BACKEND !== 'live') return () => {};
+  claimLiveProject(project?.name ?? '(unnamed project)');
   const { reentrant } = acquireLock();
   try {
     if (process.env.MIDNIGHT_LIVE_ALLOW_DIRTY !== '1') await assertFreshNode();

contracts/test-utils/harness/liveProgressReporter.ts

@@ -10,6 +10,11 @@ import type { Reporter, TestCase, TestModule } from 'vitest/node';
  * worker in `live.setup` (the only place that knows its `VITEST_POOL_ID`). The
  * total accrues as modules are collected, so the first few lines may show a
  * smaller denominator until collection finishes.
+ *
+ * A SKIPPED test never runs `beforeEach`, so no worker stamps it. Rather than
+ * print `[w?]`, fall back to the last worker seen for that test's module: the
+ * module was loaded and run by that worker even where an individual test was
+ * skipped. `?` remains only for a module where nothing ever reported a worker.
  */
 const MARKS: Record<string, string> = {
   passed: '✓',
@@ -20,10 +25,13 @@ const MARKS: Record<string, string> = {
 export default class LiveProgressReporter implements Reporter {
   private total = 0;
   private done = 0;
+  /** module id → the last worker that reported a test from it. */
+  private workerByModule = new Map<string, number>();
 
   onTestRunStart(): void {
     this.total = 0;
     this.done = 0;
+    this.workerByModule.clear();
   }
 
   onTestModuleCollected(module: TestModule): void {
@@ -34,7 +42,10 @@ export default class LiveProgressReporter implements Reporter {
     const { state } = testCase.result();
     if (state === 'pending') return; // not finished yet
     this.done += 1;
-    const worker = (testCase.meta() as { workerId?: number }).workerId ?? '?';
+    const moduleId = testCase.module.moduleId;
+    const stamped = (testCase.meta() as { workerId?: number }).workerId;
+    if (stamped !== undefined) this.workerByModule.set(moduleId, stamped);
+    const worker = stamped ?? this.workerByModule.get(moduleId) ?? '?';
     const mark = MARKS[state] ?? '·';
     const ms = Math.round(testCase.diagnostic()?.duration ?? 0);
     console.log(

…dentialFungibleTokenPublicSupply.compact …dentialFungibleTokenPublicSupply.compactcontracts/test/integration/_mocks/ConfidentialFungibleTokenPublicSupply.compact renamed to contracts/test/integration/_mocks/ComposedConfidentialFungibleTokenPublicSupply.compact contracts/test/integration/_mocks/ConfidentialFungibleTokenPublicSupply.compact renamed to contracts/test/integration/_mocks/ComposedConfidentialFungibleTokenPublicSupply.compact

@@ -17,6 +17,10 @@
 // "Public" supply means the total, and therefore each mint/burn delta, is
 // disclosed on chain. Balances stay confidential; only the aggregate is
 // visible.
+//
+// The `Composed` prefix is load-bearing: compactc keys each `artifacts/<name>/`
+// directory on the source basename, so this file must not share a basename with
+// the `src/token/extensions` extension it composes.
 pragma language_version >= 0.23.0;
 
 import CompactStandardLibrary;

contracts/test/integration/fixtures/confidentialFungibleTokenPublicSupply.ts

@@ -7,7 +7,7 @@ import {
   ledger,
   Contract as MockConfidentialFungibleTokenPublicSupply,
   type Token_EscrowEntry,
-} from '../../../artifacts/ConfidentialFungibleTokenPublicSupply/contract/index.js';
+} from '../../../artifacts/ComposedConfidentialFungibleTokenPublicSupply/contract/index.js';
 import {
   ConfidentialFungibleTokenPrivateState,
   ConfidentialFungibleTokenWitnesses,
@@ -16,7 +16,8 @@ import {
 
 /**
  * Integration fixture for the assembled ConfidentialFungibleToken + PublicSupply
- * contract (`test/integration/_mocks/ConfidentialFungibleTokenPublicSupply`).
+ * contract
+ * (`test/integration/_mocks/ComposedConfidentialFungibleTokenPublicSupply`).
  *
  * It reuses the base token's confidential-token witnesses and private state (SK,
  * EK, plaintext cache, randomness seed) verbatim — the assembled contract's
@@ -45,7 +46,7 @@ const Base = createSimulator<
   contractArgs: (name, symbol, decimals) => [name, symbol, decimals],
   ledgerExtractor: (state) => ledger(state),
   witnessesFactory: () => ConfidentialFungibleTokenWitnesses(),
-  artifactName: 'ConfidentialFungibleTokenPublicSupply',
+  artifactName: 'ComposedConfidentialFungibleTokenPublicSupply',
 });
 
 export class ConfidentialFungibleTokenPublicSupplySimulator extends Base {