fix(deployer): unblock preprod wallet sync

preprod's dust history is a ~1M-event global stream that every client
replays on sync. Deploying any contract there failed before a tx was
ever built: the dust wallet OOMed mid-replay, and once that was worked
around the sync gate never completed. Both trace to the pre-fix
wallet-sdk-dust-wallet@4.0.0 pin (midnightntwrk/midnight-wallet#425).

* wallet/handler: set batchUpdates = { size: 5000, timeout: 1,
  spacing: 4 } on the shared wallet config so the fresh-sync and the
  cache-restore paths (shielded + dust alike) stream the replay in
  larger chunks instead of exhausting the V8 heap at the SDK default
  batch size of 10.
* deployer: replace the strict FacadeState.isSynced tip gate with
  isCompleteWithin(50) on each sub-wallet. On a live chain the dust
  stream advances continuously, so strict completion never fires and
  the gate timed out on a fully usable wallet. The gate still waits on
  all three sub-wallets to avoid the custom-error-170 regression.
* cli + README: document the NODE_OPTIONS max-old-space-size bump
  still useful for a first preprod sync, plus the tolerant tip gate.

Verified on preprod: ShieldedFungibleToken dry-run synced to tip in
~1m12s from a warm cache (dust ~1.08M events, no OOM) and the gate
completed even though dust never reached strict completion.

Closes #115

Author: 0xisk

packages/cli/src/runDeploy.ts

@@ -299,6 +299,21 @@ function showUsage(): void {
   console.log(
     chalk.yellow('  compact-deploy Token --network preprod --dry-run --json'),
   );
+  console.log(
+    chalk.yellow(
+      '\nNote: a first sync on a long-history network (e.g. preprod) can exceed',
+    ),
+  );
+  console.log(
+    chalk.yellow(
+      "      Node's default heap. On 'JavaScript heap out of memory', raise it:",
+    ),
+  );
+  console.log(
+    chalk.yellow(
+      '      NODE_OPTIONS=--max-old-space-size=8192 compact-deploy <Contract> …',
+    ),
+  );
 }
 
 function packageVersion(): string {

packages/deployer/README.md

@@ -45,6 +45,7 @@ Exit codes: `0` ok · `2` config error · `3` wallet error · `4` provider unrea
 - **First sync is slow** (~3 min on preview, 30–60 min on preprod from genesis). Cache makes reruns near-instant.
 - **Bump sync timeout**: `--sync-timeout 3600` (default 10 min).
 - **Bump Node heap** for long-history chains: `NODE_OPTIONS="--max-old-space-size=8192"`.
+- **Tip gate is tolerant**: sync completes once every sub-wallet is within 50 events of the tip, not at an exact gap of 0. On a live network the global dust stream advances continuously, so an exact-match gate would never fire.
 - **Seed source**: `--seed-file`, `MN_DEPLOYER_SEED`, or `[wallet].keystore`. The `wallet = { source = "local" }` shorthand is dev-preset only.
 
 ## Wallet cache
@@ -153,7 +154,7 @@ signing_key_file = "./deploy/Vault.signingkey"
 
 4. **Dust fee overhead default breaks faucet wallets.** testkit-js default `additionalFeeOverhead` is `5e20` vs a faucet wallet's `~3e15` dust → `Insufficient Funds: could not balance dust`. Deployer overrides to `5e14` for non-mainnet. Library users constructing their own provider must mirror this.
 
-5. **Long-history dust sync exhausts default Node heap.** Use `NODE_OPTIONS="--max-old-space-size=16384"` for the first sync. Cache fixes subsequent runs.
+5. **Long-history dust sync exhausts default Node heap.** The deployer now raises the dust/shielded sync batch size (`batchUpdates = { size: 5000, … }`) so the replay no longer OOMs mid-stream on `wallet-sdk-dust-wallet@4.0.0` ([midnightntwrk/midnight-wallet#425](https://github.com/midnightntwrk/midnight-wallet/issues/425)). The restored dust tree plus shielded trial-decryption can still spike past V8's ~2 GB default old-space on a first preprod sync, so set `NODE_OPTIONS="--max-old-space-size=8192"` for that run. Cache fixes subsequent runs.
 
 ## Programmatic API
 

packages/deployer/src/deployer.test.ts

@@ -129,14 +129,27 @@ function fakeProvider(coinKey = '0xCOIN'): FakeProvider {
     isSynced: true,
     shielded: {
       balances: anyKeyHasBalance,
-      state: { progress: { isStrictlyComplete: () => true } },
+      state: {
+        progress: {
+          isStrictlyComplete: () => true,
+          isCompleteWithin: () => true,
+        },
+      },
     },
     unshielded: {
       balances: anyKeyHasBalance,
-      progress: { isStrictlyComplete: () => true },
+      progress: {
+        isStrictlyComplete: () => true,
+        isCompleteWithin: () => true,
+      },
     },
     dust: {
-      state: { progress: { isStrictlyComplete: () => true } },
+      state: {
+        progress: {
+          isStrictlyComplete: () => true,
+          isCompleteWithin: () => true,
+        },
+      },
       balance: () => 1n,
     },
   };
@@ -213,14 +226,27 @@ function syncedState(
     isSynced: true,
     shielded: {
       balances: shielded,
-      state: { progress: { isStrictlyComplete: () => true } },
+      state: {
+        progress: {
+          isStrictlyComplete: () => true,
+          isCompleteWithin: () => true,
+        },
+      },
     },
     unshielded: {
       balances: unshielded,
-      progress: { isStrictlyComplete: () => true },
+      progress: {
+        isStrictlyComplete: () => true,
+        isCompleteWithin: () => true,
+      },
     },
     dust: {
-      state: { progress: { isStrictlyComplete: () => true } },
+      state: {
+        progress: {
+          isStrictlyComplete: () => true,
+          isCompleteWithin: () => true,
+        },
+      },
       balance: () => 0n,
     },
   };
@@ -414,7 +440,7 @@ describe('Deployer', () => {
   });
 
   describe('syncAndVerifyFunds (owned-wallet branch)', () => {
-    it('should reject with a timeout error when the wallet never reports isSynced', async () => {
+    it('should reject with a timeout error when the wallet never reaches chain tip', async () => {
       const built = fakeOwnedFromProvider(
         fakeProviderWithState(Rx.NEVER, '0xSTUCK'),
       );
@@ -430,6 +456,110 @@ describe('Deployer', () => {
       ).rejects.toThrow(/Wallet sync timeout after 50ms/);
     });
 
+    it('should complete sync when sub-wallets are within the gap but not strictly complete', async () => {
+      // Live-chain shape (issue #115): the global dust stream keeps
+      // advancing, so dust is never strictly complete (gap 0) but settles
+      // within the tolerated gap. The old strict `isSynced` gate would hang
+      // here forever; the tolerant `isCompleteWithin` gate must proceed.
+      const anyBal = new Proxy({} as Record<string, bigint>, {
+        get: () => 1n,
+      });
+      const liveTipState = {
+        isSynced: false,
+        shielded: {
+          balances: anyBal,
+          state: {
+            progress: {
+              isStrictlyComplete: () => true,
+              isCompleteWithin: () => true,
+            },
+          },
+        },
+        unshielded: {
+          balances: anyBal,
+          progress: {
+            isStrictlyComplete: () => true,
+            isCompleteWithin: () => true,
+          },
+        },
+        dust: {
+          state: {
+            progress: {
+              isStrictlyComplete: () => false,
+              isCompleteWithin: () => true,
+            },
+          },
+          balance: () => 1n,
+        },
+      };
+      const built = fakeOwnedFromProvider(
+        fakeProviderWithState(Rx.of(liveTipState as unknown), '0xLIVE-TIP'),
+      );
+      vi.mocked(WalletHandler.build).mockResolvedValueOnce(built.owned);
+      await using d = await Deployer.prepare({
+        contract: 'Counter',
+        network: 'local',
+        configPath: fx.configPath,
+        logger: silentLogger,
+        syncTimeoutMs: 1000,
+      });
+      expect(d.deployer).toBe('0xLIVE-TIP');
+    });
+
+    it('should NOT complete sync while any sub-wallet is outside the gap', async () => {
+      // Dust still outside the tolerated gap: the gate must keep waiting and
+      // ultimately time out rather than deploy against a half-synced wallet.
+      const anyBal = new Proxy({} as Record<string, bigint>, {
+        get: () => 1n,
+      });
+      const laggingState = {
+        isSynced: false,
+        shielded: {
+          balances: anyBal,
+          state: {
+            progress: {
+              isStrictlyComplete: () => true,
+              isCompleteWithin: () => true,
+            },
+          },
+        },
+        unshielded: {
+          balances: anyBal,
+          progress: {
+            isStrictlyComplete: () => true,
+            isCompleteWithin: () => true,
+          },
+        },
+        dust: {
+          state: {
+            progress: {
+              isStrictlyComplete: () => false,
+              isCompleteWithin: () => false,
+            },
+          },
+          balance: () => 1n,
+        },
+      };
+      // Emit the lagging state, then hang: the gate filters it out and must
+      // keep waiting (not complete the sequence) so the timeout can fire.
+      const built = fakeOwnedFromProvider(
+        fakeProviderWithState(
+          Rx.concat(Rx.of(laggingState as unknown), Rx.NEVER),
+          '0xLAGGING',
+        ),
+      );
+      vi.mocked(WalletHandler.build).mockResolvedValueOnce(built.owned);
+      await expect(
+        Deployer.prepare({
+          contract: 'Counter',
+          network: 'local',
+          configPath: fx.configPath,
+          logger: silentLogger,
+          syncTimeoutMs: 50,
+        }),
+      ).rejects.toThrow(/Wallet sync timeout after 50ms/);
+    });
+
     it('should throw UnfundedWalletError when shielded and unshielded balances are both empty', async () => {
       const built = fakeOwnedFromProvider(
         fakeProviderWithState(Rx.of(syncedState({}, {})), '0xEMPTY'),
@@ -684,17 +814,18 @@ describe('Deployer', () => {
 
   describe('describeProgress branches', () => {
     it('should render the progress percentage when highest > 0', async () => {
-      // Mid-sync state (NOT yet `isSynced`) that drives the progress
+      // Mid-sync state (still short of the tip) that drives the progress
       // subscription's "else" branch (highest > 0). Then a follow-up
-      // synced state lets `firstValueFrom(filter(isSynced))` resolve so
-      // the prepare call terminates.
+      // tip-reached state lets the `isCompleteWithin` gate resolve so the
+      // prepare call terminates.
       const midState = {
         isSynced: false,
         shielded: {
           balances: {} as Record<string, bigint>,
           state: {
             progress: {
               isStrictlyComplete: () => false,
+              isCompleteWithin: () => false,
               appliedIndex: 10n,
               highestIndex: 100n,
               isConnected: true,
@@ -705,6 +836,7 @@ describe('Deployer', () => {
           balances: {} as Record<string, bigint>,
           progress: {
             isStrictlyComplete: () => false,
+            isCompleteWithin: () => false,
             appliedId: 5n,
             highestTransactionId: 50n,
             isConnected: true,
@@ -714,6 +846,7 @@ describe('Deployer', () => {
           state: {
             progress: {
               isStrictlyComplete: () => false,
+              isCompleteWithin: () => false,
               appliedIndex: 1n,
               highestIndex: 10n,
               isConnected: true,
@@ -729,14 +862,27 @@ describe('Deployer', () => {
         isSynced: true,
         shielded: {
           balances: anyKeyHasBalance,
-          state: { progress: { isStrictlyComplete: () => true } },
+          state: {
+            progress: {
+              isStrictlyComplete: () => true,
+              isCompleteWithin: () => true,
+            },
+          },
         },
         unshielded: {
           balances: anyKeyHasBalance,
-          progress: { isStrictlyComplete: () => true },
+          progress: {
+            isStrictlyComplete: () => true,
+            isCompleteWithin: () => true,
+          },
         },
         dust: {
-          state: { progress: { isStrictlyComplete: () => true } },
+          state: {
+            progress: {
+              isStrictlyComplete: () => true,
+              isCompleteWithin: () => true,
+            },
+          },
           balance: () => 1n,
         },
       };

packages/deployer/src/deployer.ts

@@ -29,6 +29,19 @@ import {
  */
 const DEFAULT_SYNC_TIMEOUT_MS = 10 * 60 * 1000;
 
+/**
+ * Tolerated sync gap, in events, for the chain-tip gate.
+ * `FacadeState.isSynced` requires every sub-wallet to be *strictly*
+ * complete (gap 0). On a live network the global dust stream advances
+ * continuously, so the dust wallet sits a few events behind the tip
+ * indefinitely and strict `isSynced` never flips. The gate would then time
+ * out on a wallet that is in fact fully usable. `isCompleteWithin(50)` is the
+ * SDK default (and what the unshielded wallet uses internally): it treats
+ * "within 50 events of tip" as synced, which a live wallet reaches and
+ * holds. See OpenZeppelin/compact-tools#115.
+ */
+const SYNC_MAX_GAP = 50n;
+
 import { ConstructorArgs } from './loaders/args.ts';
 import { Artifact } from './loaders/artifact.ts';
 import { InitialPrivateState } from './loaders/init-state.ts';
@@ -469,11 +482,15 @@ function describeProgress(p: { isStrictlyComplete: () => boolean }): string {
 }
 
 /**
- * Drive the wallet to chain tip and assert spendable funds. Uses
- * `state.isSynced` (strict-complete on all three sub-wallets) as the
- * gate. Looser gates regressed on local with `Invalid Transaction
- * (custom error 170)`. Throttles progress logs to once per 30 s.
- * Throws {@link UnfundedWalletError} on empty wallet.
+ * Drive the wallet to chain tip and assert spendable funds. Gates on each
+ * sub-wallet being within {@link SYNC_MAX_GAP} events of the tip rather
+ * than strictly complete: on a live network the global dust stream never
+ * settles to gap 0, so strict `FacadeState.isSynced` never fires and the
+ * gate times out on a perfectly usable wallet (#115). The gate still waits
+ * on all three sub-wallets; dropping the dust/shielded wait entirely
+ * regressed on local with `Invalid Transaction (custom error 170)`.
+ * Throttles progress logs to once per 30 s. Throws
+ * {@link UnfundedWalletError} on empty wallet.
  */
 async function syncAndVerifyFunds(args: {
   wallet: MidnightWalletProvider;
@@ -577,7 +594,15 @@ async function syncAndVerifyFunds(args: {
   try {
     synced = await Rx.firstValueFrom(
       state$.pipe(
-        Rx.filter((s: FacadeState) => s.isSynced),
+        // Tolerant tip gate: all three sub-wallets within SYNC_MAX_GAP
+        // events of the tip. Strict `s.isSynced` never fires on a live
+        // chain because the global dust stream keeps advancing (#115).
+        Rx.filter(
+          (s: FacadeState) =>
+            s.shielded.state.progress.isCompleteWithin(SYNC_MAX_GAP) &&
+            s.dust.state.progress.isCompleteWithin(SYNC_MAX_GAP) &&
+            s.unshielded.progress.isCompleteWithin(SYNC_MAX_GAP),
+        ),
         Rx.timeout({
           each: timeoutMs,
           with: () =>

packages/deployer/src/wallet/handler.test.ts

@@ -210,6 +210,32 @@ describe('WalletHandler', () => {
     });
   });
 
+  describe('sync batching', () => {
+    it('should set a large batchUpdates size on the shared config for both sub-wallets', async () => {
+      wireTestkitChain(fakeProvider());
+      await WalletHandler.build(logger, fakeEnv('preprod'), {
+        kind: 'hex',
+        value: '00',
+      });
+      // The shared config is mutated in place and threaded into every
+      // sub-wallet factory, so asserting on the dust + shielded calls proves
+      // the OOM workaround (issue #115) covers both. Default SDK batch size
+      // is 10, which OOMs replaying preprod's ~1M-event dust stream.
+      const withBatch = expect.objectContaining({
+        batchUpdates: { size: 5000, timeout: 1, spacing: 4 },
+      });
+      expect(WalletFactory.createDustWallet).toHaveBeenCalledWith(
+        withBatch,
+        expect.any(Uint8Array),
+        expect.anything(),
+      );
+      expect(WalletFactory.createShieldedWallet).toHaveBeenCalledWith(
+        withBatch,
+        expect.any(Uint8Array),
+      );
+    });
+  });
+
   describe('provider wiring', () => {
     it('should expose the wallet built by MidnightWalletProvider.withWallet via .provider', async () => {
       const provider = fakeProvider();