Skip to content

fix(ses): cyclic star export with renaming reexport (issue #59) - refresh for #3276 feedback#379

Open
kriscendobot wants to merge 15 commits into
masterfrom
fix/issue-59-star-export-cycle
Open

fix(ses): cyclic star export with renaming reexport (issue #59) - refresh for #3276 feedback#379
kriscendobot wants to merge 15 commits into
masterfrom
fix/issue-59-star-export-cycle

Conversation

@kriscendobot

Copy link
Copy Markdown
Collaborator

Refs: endojs/endo#59, endojs/endo#3276
Supersedes: closed mirror PR #336 (branch force-pushed to address review feedback).

Description

Mirror of upstream endojs/endo#3276 (fix for the 2019 issue endojs/endo#59, cyclic star export with renaming reexport), refreshed for the bot-pushable fork so the gauntlet can re-run against the current upstream PR head.

This refresh sets the mirror head to upstream PR head f4aad15a and adds a single companion regression test that addresses inline review feedback from naugtur on upstream pullrequestreview-4388440170.

Naugtur's question

Is a situation possible where all calls to the deferring notify happen before upstreamNotify can be obtained? I was trying to come up with when that could happen and I thought of unused live bindings.

Disposition: verified, no source change

The "all calls happen before upstreamNotify is obtained" case is reachable in theory (a cyclic re-export of an unused live binding in which no further wireUp re-references the deferred notifier), but it is benign. Tracing the closure at packages/ses/src/module-instance.js:379-396:

  1. The internal notify(update) call at line 403 always fires once during wireUpExportNotifier. If upstreamInstance.notifiers[deferredImportName] is undefined at that moment, the call is queued.
  2. The closure resolves only on a subsequent call. A subsequent call arrives if (a) a module higher in the chain re-exports through us (via export * or another renaming reexport), or (b) a downstream consumer subscribes through the main imports() updateRecord loop at line 449.
  3. If neither (a) nor (b) happens, the closure stays pending and the local value stays undefined. The export's getter at line 404-411 then returns undefined.
  4. The unused-live-binding case (export var y; declared but never assigned) is exactly this shape under Node.js too: every projection of the cycle reads as undefined. Verified directly with node.

The new test in packages/ses/test/import-gauntlet.test.js (cyclic star export with renaming reexport, unused live binding) pins SES's behavior to Node.js's reference behavior for this case.

Why this is the right shape

The deferring closure's contract is "forward to the upstream's notifier once it exists, otherwise queue". The contract is well-defined whether or not a second call ever arrives; the queue is bounded by the number of subscribers (zero in the unused case). The "all calls before upstream" hypothetical is just the zero-subscriber subcase; it does not mis-route any update.

Verification

  • yarn workspace ses test 503 pass + 2 known failures + 2 skipped (includes the new gauntlet test).
  • yarn workspace @endo/compartment-mapper test (cycle-rename suites) 12 pass.
  • Node.js parity for the unused-binding fixture verified outside the test runner.

Security Considerations

None. Test-only addition; no source change.

Scaling Considerations

None.

Documentation Considerations

None.

Testing Considerations

The new test exercises the unused-live-binding subcase and pins it to Node.js parity. Removing the deferring branch makes the original (issue #59) test fail with TypeError: notify is not a function; the new test passes with or without the deferring branch (it is a parity assertion, not a defect reproducer).

Compatibility Considerations

None.

Upgrade Considerations

None.

kriskowal and others added 2 commits May 21, 2026 17:43
When a module re-exports `*` from another module, and that other module
re-exports a binding from the first under a *different* exported name
(`export { y as x } from './mod1.js'`), the linker visited the first
module while its star-imported notifier for `y` had not yet been wired.
The synchronous wireUp at the cycle's back-edge then passed `undefined`
as the upstream notifier, which manifested as `TypeError: notify is not
a function` at `packages/ses/src/module-instance.js`. The original 2019
issue described the failure as `SyntaxError: ... does not provide an
export named 'y'`; the surface symptom evolved as the linker matured,
but the underlying defect is the same.

`wireUpExportNotifier` now installs a deferred forwarding notifier for
re-exports whose upstream notifier is not yet present. The forwarding
queues subscribers until the upstream resolves, then drains them
through. By the time any module downstream subscribes to the re-export,
the upstream module has completed its candidate-all walk and the
upstream notifier exists, so the chain converges.

Two regression surfaces accompany the fix. In SES, `import-gauntlet`
adds `cyclic star export with renaming reexport (issue #59)` exercising
the exact reproducer from the issue against the SES linker directly. In
the compartment mapper, a three-module fixture (`star-reexporter` re-
exports `*` from `export-renamer`; `export-renamer` re-exports `y as x`
from `star-reexporter`) drives the same shape through `loadLocation`,
`importLocation`, the archive round-trip pair, and `makeArchiveFromMap`,
plus a Node.js parity test that imports the same fixture under plain
Node.js (no SES, no compartment mapper) and asserts identical expected
values from a shared assertions module. Pinning the compartment
mapper's behavior to Node.js's reference behavior keeps the expected
values defined in one place and teases linker behavior out of SES
rather than asserting it against itself.

Reverting the `wireUpExportNotifier` change while keeping either test
surface reproduces `TypeError: notify is not a function` (nine of the
eleven compartment-mapper import-path variants fail; the two archive-
integrity variants pass because they do not import the fixture; the
Node.js parity test is unaffected). Re-applying the fix restores all
twelve passing tests.

Fixes #59
Companion regression for endojs/endo#59 addressing review feedback on
endojs/endo#3276: a sibling fixture where the live binding `y` is
declared but never assigned. Node.js reads every projection of the
cycle as `undefined` for this shape (verified directly with `node`),
so the SES linker must match. The deferring closure introduced by the
fix either resolves (when a wireUp higher in the chain re-references
the binding) or stays pending; the namespace reads agree with Node.js
in either case.

This commit adds executable evidence; no source change.
@kriscendobot

Copy link
Copy Markdown
Collaborator Author

@naugtur Thanks for the careful read on upstream endojs/endo#3276.

Disposition: verified, no source change.

Tracing the deferring closure at packages/ses/src/module-instance.js:379-396: the "all calls happen before upstreamNotify is obtained" case is reachable (a cyclic re-export of an unused live binding, no higher wireUp re-references the deferred notifier, no downstream subscriber), but it is benign. In that subcase the queue stays full, the local value stays undefined, and the export reads as undefined. Node.js reads the same shape (export var y; declared but never assigned) as undefined for every projection of the cycle.

Companion regression cyclic star export with renaming reexport, unused live binding added at SHA 96ea2c59c pins SES's behavior to Node.js parity for the unused-binding fixture. It passes both with and without the deferring branch (parity assertion, not a defect reproducer), but it locks the contract.

Bot mirror PR #379 carries the refresh (closed mirror #336 was force-pushed beyond GitHub's reopen tolerance).

}
pendingUpdaters.length = 0;
upstreamNotify(update);
};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naugtur and I agree there is a likely opportunity for refactoring, here. The notifier system would benefit from a synchronous variant on Promise.withResolvers, e.g., makeNotifierWithResolver, that can be used here and in makeVirtualModule, to reduce the likelihood that these parallel implementations will drift.

Naugtur also asked for a test that verifies similar behavior if the reexporting module is CommonJS. I would like a test that at least verifies that the behavior is consistent with Node.js in that scenario.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied makeNotifierWithResolver to the wireUpExportNotifier deferred branch in this commit. The Naugtur-asked CommonJS reexporter test landed in packages/ses/test/import-cjs.test.js, pinned to Node.js parity.

For makeVirtualModuleInstance the builder's design judgment was to leave the existing pattern in place: that site is a live-cell fan-out where set is called repeatedly and every subscriber must receive every future value, while makeNotifierWithResolver is a one-shot redirect (Promise.withResolvers shape). Unifying would either need a "repeat-resolve" semantic or a current-value protocol — either direction seemed to obscure local semantics. Happy to revisit if you'd like the unification forced or want a different abstraction shape (e.g., a separate makeLiveNotifier for the fan-out site).

endolinbot added 2 commits June 2, 2026 03:54
Introduce a synchronous variant of Promise.withResolvers. The helper
returns { notify, resolve }: subscribers attached via notify(update)
before resolve(targetNotify) is called are queued; once resolve is
called, queued updaters are replayed to the target notifier and
subsequent notify(update) calls forward directly through. resolve is
one-shot (idempotent for repeat calls), which lets a caller invoke it
lazily on each notify and have only the first invocation take effect.

Apply the helper to the cycle-resolver branch of wireUpExportNotifier
in module-instance.js, replacing the inline pendingUpdaters[] +
resolvedUpstreamNotify state machine. The two patterns are now
syntactically separated from their call site, reducing the chance that
the local state machine will drift away from any future second use.

Refs: endojs/endo#3276 (kriskowal review)
Add a regression test in import-cjs.test.js verifying that a cyclic
star-export topology in which the "reexporter" is a CommonJS module
behaves consistent with Node.js. Node.js rejects ESM-in-CJS-cycle
outright (ERR_REQUIRE_CYCLE_MODULE), so the parity comparison is
against the pure-CJS cycle: snapshot-at-call-time semantics for the
CJS side (property capture sees whatever the renamer had assigned by
the re-entry instant), live-binding semantics for the ESM side. Both
namespaces project the same shapes the test pins.

Addresses naugtur's review feedback on endojs/endo#3276 asking for a
test of the cyclic star-export with a CommonJS reexporting module.

Refs: endojs/endo#3276 (naugtur review)
Comment thread packages/ses/test/import-gauntlet.test.js Outdated
Comment thread packages/ses/test/import-cjs.test.js Outdated
Comment thread packages/ses/test/import-cjs.test.js Outdated
endolinbot added 3 commits June 2, 2026 04:42
… tests (#59 follow-up)

Mirror the cycle-rename parity-test layout for the pure-CommonJS cyclic
reexporter scenario, in which Node.js and SES agree. The fixture under
fixtures-cycle-cjs-reexporter/node_modules/app/ expresses the
star-reexporter and renaming reexporter as on-disk .cjs modules with a
live getter for the renamed export. cycle-cjs-reexporter.test.js runs
the fixture through the compartment-mapper scaffold;
cycle-cjs-reexporter-node-parity.test.js runs the same fixture under
plain Node.js. Both tests assert through the shared
_cycle-cjs-reexporter-assertions.js module, so parity is verified by
construction: if both tests pass, the compartment mapper's CommonJS
cycle behavior matches Node.js for this case.
…follow-up)

Add a parity-test pair that programmatically verifies the divergence
between Node.js and SES for the ESM-in-CommonJS-cycle topology. The
fixture under fixtures-cycle-esm-in-cjs/node_modules/app/ has a CJS
bridge module that require()s an ESM peer module that imports back from
the bridge. cycle-esm-in-cjs.test.js asserts SES allows the topology
and the namespace projects the live binding (bridgeValue === 42).
cycle-esm-in-cjs-node-parity.test.js spawns a fresh Node.js process on
the same fixture and asserts Node rejects with
ERR_REQUIRE_CYCLE_MODULE. Together the two tests pin the divergence as
a verified property rather than narrative prose.
…tment-mapper parity

Rewrite the JSDoc on the cyclic CommonJS reexporter test in
import-cjs.test.js and the companion unused-live-binding test in
import-gauntlet.test.js so each block is primarily an explanation of
what the test verifies (the shapes of the projected namespaces, the
specific snapshot vs live-binding distinction, the parity property),
not the procedural history of how the test came to be. The in-process
SES regression is retained on the import-cjs.test.js side because it
exercises the module-instance linker directly through the Compartment
API with inline ModuleSources, a path the compartment-mapper parity
suite does not cover. The prose now points at the parity suite for the
parity-with-Node substantiation:
packages/compartment-mapper/test/cycle-cjs-reexporter.test.js and its
node-parity sibling for the pure-CommonJS agreement case;
packages/compartment-mapper/test/cycle-esm-in-cjs.test.js and its
node-parity sibling for the ESM-in-CJS-cycle divergence
(ERR_REQUIRE_CYCLE_MODULE on Node, allowed on SES).
@kriscendobot

Copy link
Copy Markdown
Collaborator Author

Addressed all three asks in commits 4d4953d..f89afdb (head 4d4953d):

  • r3338677487 (reframe prose + commit a parity test): Reframed JSDoc on the CJS reexporter test in import-cjs.test.js and on the unused-live-binding test in import-gauntlet.test.js to focus on what the tests verify rather than the procedural impetus. Committed shared-fixture parity infrastructure at packages/compartment-mapper/test/cycle-cjs-reexporter{,−node-parity}.test.js + fixtures-cycle-cjs-reexporter/ + _cycle-cjs-reexporter-assertions.js — mirrors the existing cycle-rename parity layout.

  • r3338682426 (state SES/Node divergence programmatically): Added packages/compartment-mapper/test/cycle-esm-in-cjs{,−node-parity}.test.js + fixtures-cycle-esm-in-cjs/. The Node parity test spawns a fresh Node process on the fixture and asserts stderr contains ERR_REQUIRE_CYCLE_MODULE; the SES test asserts SES's actual non-throwing behavior. Divergence is now a verified property rather than narrative.

  • r3338685696 (parity-test concept for gardener docs): Sent message to the garden's gardener role to document the parity-test pattern (shared fixture + shared assertions + SES test + Node parity test, covering both convergence and divergence shapes) as a citable norm for future PRs. Reference implementations and pattern detail in the gardener inbox.

yarn test clean in both packages/ses and packages/compartment-mapper; yarn build:types:check exit 0; git grep -E 'Naugtur asked|verified directly with .node.' returns zero matches.

…59 follow-up)

Add the parity test pair kriskowal asked for on #379
(review comment 3338677487): the unused-live-binding shape of the cyclic
star-export with renaming reexport (endojs/endo#59) needs a parity test
substantiating the Node.js parity claim, ideally with a shared fixture as
with cycle-rename parity. The companion populated-binding shape was already
covered by cycle-rename.test.js and cycle-rename-node-parity.test.js through
_cycle-rename-assertions.js; this commit lands the analogous trio for the
unused-live-binding shape:

  fixtures-cycle-rename-unused/node_modules/app/ - three modules; the
    renamer's `export var y` has no initializer.
  _cycle-rename-unused-assertions.js - shared assertion module; expected
    projections are { x: undefined, y: undefined } and captured: undefined.
  cycle-rename-unused.test.js - compartment-mapper scaffold exercise.
  cycle-rename-unused-node-parity.test.js - Node.js exercise of the same
    fixture; parity is verified by construction when both pass.

Also update the in-process SES regression's prose in
packages/ses/test/import-gauntlet.test.js to reference the new parity pair,
matching the cross-reference pattern the populated-binding test already
uses for cycle-rename.

Refs: endojs/endo#3276 (naugtur), #379 (kriskowal)
@kriscendobot

Copy link
Copy Markdown
Collaborator Author

Audit summary: feedback completeness on endojs/endo#3276 + #379

This is an audit-only sweep across naugtur's three inline asks on endojs/endo#3276 and kriskowal's five mirror-side asks on this PR. Each item is verified against the current head (f1a7dfb60).

naugtur, endojs/endo#3276 inline asks

  1. (packages/ses/src/module-instance.js:379, comment 3323491701) "Is a situation possible where all calls to the deferring notify happen before upstreamNotify can be obtained? ... unused live bindings."
    Addressed. packages/ses/test/import-gauntlet.test.js carries the unused-live-binding regression at cyclic star export with renaming reexport, unused live binding: the renamer's export var y has no initializer, so the deferring notifier on x queues subscribers but the upstream is never updated; every projection of the cycle reads undefined. Verified against Node.js. Pinned in commit 96ea2c59c and substantiated with a shared-fixture parity pair in commit f1a7dfb60 (see ask 3 below).

  2. (packages/compartment-mapper/test/fixtures-cycle-rename/node_modules/app/main.js, comment 3323503838) "Would it still work if the cycle was longer and a cjs module was involved in it?"
    Addressed. Two three-module fixtures land on the mirror covering the longer-cycle-with-CJS shape:

    • All-CommonJS three-module reexporter cycle: packages/compartment-mapper/test/fixtures-cycle-cjs-reexporter/ (commit f89afdb78), exercised through cycle-cjs-reexporter.test.js and pinned to Node.js via cycle-cjs-reexporter-node-parity.test.js. Shared assertions in _cycle-cjs-reexporter-assertions.js.
    • ESM-in-CJS three-module cycle: packages/compartment-mapper/test/fixtures-cycle-esm-in-cjs/ (commit 340479b2e), exercised through cycle-esm-in-cjs.test.js (SES allows) and cycle-esm-in-cjs-node-parity.test.js (Node rejects with ERR_REQUIRE_CYCLE_MODULE). This is the divergence case.
  3. (packages/ses/src/module-instance.js, comment 3323524839) "shared notifier primitive between makeModuleInstance and makeVirtualModuleInstance to reduce the risk of interop failure."
    Partial. makeNotifierWithResolver is extracted into packages/ses/src/notifier-with-resolver.js (commit 6b80ac3ee) and applied to makeModuleInstance.wireUpExportNotifier. It is not applied to makeVirtualModuleInstance; the design judgment (recorded on the inline reply, comment 3338583474) is that the wireUp site is a one-shot redirect (Promise.withResolvers shape) while makeVirtualModuleInstance is a live-cell fan-out where every subscriber must receive every future value, and unifying would either need a "repeat-resolve" semantic or a current-value protocol. The reply offers a separate makeLiveNotifier shape if the unification is wanted.

kriskowal, #379 inline asks

  1. (packages/ses/src/module-instance.js:389, comment 3338365530) Same as naugtur chore: bump actions/setup-python from 5.6.0 to 6.2.0 #3 plus "a test that at least verifies that the behavior is consistent with Node.js in that scenario."
    Partial / addressed. The refactor is partially applied per ask 3 above. The Node.js parity scenario for the CJS reexporter is covered by cycle-cjs-reexporter-node-parity.test.js plus shared _cycle-cjs-reexporter-assertions.js.

  2. (Comment 3338583474) Bot's own reply, not actionable.

  3. (packages/ses/test/import-gauntlet.test.js, comment 3338677487) "Please reframe as primarily an explanation of the test rather than emphasizing the procedural impetus. Also, please commit and refer to a test that substantiates the claim that the Node.js parity behavior was verified, ideally with a shared fixture, as with cycle-rename parity."
    Addressed. Prose reframed in commit 4d4953dcb. The shared-fixture parity pair for the unused-live-binding shape was missing; commit f1a7dfb60 lands it:

    • packages/compartment-mapper/test/fixtures-cycle-rename-unused/node_modules/app/ — fixture with export var y; (no initializer).
    • packages/compartment-mapper/test/_cycle-rename-unused-assertions.js — shared assertion module; expected projections are { x: undefined, y: undefined } and captured: undefined.
    • packages/compartment-mapper/test/cycle-rename-unused.test.js — compartment-mapper scaffold exercise.
    • packages/compartment-mapper/test/cycle-rename-unused-node-parity.test.js — Node.js exercise of the same fixture; parity is verified by construction when both pass.
    • The in-process SES regression's prose now references the new parity pair, matching the cross-reference pattern the populated-binding test already uses for cycle-rename.
  4. (packages/ses/test/import-cjs.test.js, comment 3338682426) "SES does not have Node.js parity and we should state and verify this claim plainly and programmatically."
    Addressed. The divergence is stated plainly in the JSDoc on cyclic star-export with CommonJS reexporter (lines 670 onward) and verified programmatically by the cycle-esm-in-cjs parity pair: SES allows the topology and pins the namespace shape; Node spawns and asserts ERR_REQUIRE_CYCLE_MODULE. Together they verify the divergence by construction.

  5. (packages/ses/test/import-cjs.test.js, comment 3338685696) Two parts.

    • "Parity claims should be substantiated with parity tests." Addressed for every parity claim now in the PR; each is paired with a shared-fixture / Node-side / SES-side trio: cycle-rename, cycle-rename-unused (new), cycle-cjs-reexporter, and the divergence pair cycle-esm-in-cjs.
    • "Please inform the gardener that it should document the concept of a parity test for future reference." Queued as a follow-up: a message entry has been journaled to the gardener describing the parity-test pattern (shared assertion module + Node-side parity sibling + SES/scaffold-side test; both passing verifies parity by construction). The garden-side documentation work will land in a separate gardener dispatch.

CI

Post-rebase CI status will refresh as runs queue. The expected red category is test-xs (cleared as a known flake on this branch per a recent llm sync); other jobs are expected green.

kriscendobot pushed a commit to kriskowal/garden that referenced this pull request Jun 3, 2026
Audit of naugtur's 3 inline asks on endojs/endo#3276 and kriskowal's 5
mirror-side asks on endojs/endo-but-for-bots#379. Seven of eight asks
are genuinely addressed by prior commits; one ask (kriskowal #3 on
import-gauntlet.test.js) had a missing shared-fixture parity pair for
the unused-live-binding shape, closed with new compartment-mapper head
f1a7dfb60. The eighth ask (kriskowal's gardener-meta follow-up) is
queued as a message-fixer-40ac9b entry for the next gardener dispatch.

Top-level PR audit comment posted at
endojs/endo-but-for-bots#379-issuecomment-4609535322.
kriskowal added a commit to endojs/endo that referenced this pull request Jun 5, 2026
…59 follow-up)

Add the parity test pair kriskowal asked for on endojs/endo-but-for-bots#379
(review comment 3338677487): the unused-live-binding shape of the cyclic
star-export with renaming reexport (#59) needs a parity test
substantiating the Node.js parity claim, ideally with a shared fixture as
with cycle-rename parity. The companion populated-binding shape was already
covered by cycle-rename.test.js and cycle-rename-node-parity.test.js through
_cycle-rename-assertions.js; this commit lands the analogous trio for the
unused-live-binding shape:

  fixtures-cycle-rename-unused/node_modules/app/ - three modules; the
    renamer's `export var y` has no initializer.
  _cycle-rename-unused-assertions.js - shared assertion module; expected
    projections are { x: undefined, y: undefined } and captured: undefined.
  cycle-rename-unused.test.js - compartment-mapper scaffold exercise.
  cycle-rename-unused-node-parity.test.js - Node.js exercise of the same
    fixture; parity is verified by construction when both pass.

Also update the in-process SES regression's prose in
packages/ses/test/import-gauntlet.test.js to reference the new parity pair,
matching the cross-reference pattern the populated-binding test already
uses for cycle-rename.
@kriskowal

Copy link
Copy Markdown
Member

Mirror of endojs/endo#3276 (head e3f111d19).

kriskowal added a commit to endojs/endo that referenced this pull request Jun 6, 2026
…59 follow-up)

Add the parity test pair kriskowal asked for on endojs/endo-but-for-bots#379
(review comment 3338677487): the unused-live-binding shape of the cyclic
star-export with renaming reexport (#59) needs a parity test
substantiating the Node.js parity claim, ideally with a shared fixture as
with cycle-rename parity. The companion populated-binding shape was already
covered by cycle-rename.test.js and cycle-rename-node-parity.test.js through
_cycle-rename-assertions.js; this commit lands the analogous trio for the
unused-live-binding shape:

  fixtures-cycle-rename-unused/node_modules/app/ - three modules; the
    renamer's `export var y` has no initializer.
  _cycle-rename-unused-assertions.js - shared assertion module; expected
    projections are { x: undefined, y: undefined } and captured: undefined.
  cycle-rename-unused.test.js - compartment-mapper scaffold exercise.
  cycle-rename-unused-node-parity.test.js - Node.js exercise of the same
    fixture; parity is verified by construction when both pass.

Also update the in-process SES regression's prose in
packages/ses/test/import-gauntlet.test.js to reference the new parity pair,
matching the cross-reference pattern the populated-binding test already
uses for cycle-rename.

@kriskowal kriskowal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kriscendobot Please create additional tests that vary by whether the renamer or star exporter are imported first from main.js, and also vary by whether the binding is a const, let, or var. In all cases that it is possible, attempt to observe the value after it has been linked with its cyclic module but before it has been assigned or bound. This may be an observation of an exception while the binding is temporally dead (TDZ).

@kriskowal

Copy link
Copy Markdown
Member

@kriscendobot I believe I now understand ZB’s comment endojs/endo#3276 (comment): Please consider whether we can create all of the notifiers for a module instance at time of construction instead of time of link. This may require a more deliberate separation of the instantiation and link phases, but I suspect we already have a hard enough line. This would obviate deferred notifier linkage, since notifiers would always be available up front. If not, please explain why such an arrangement is not possible or how the calling convention for precompiled ModuleSource instances would need to change. It might be that the notifiers need to be partially applied before full initialization.

Please also write a test that demonstrates the same failure mode of the cyclic export, but with a named reexport rather than a star reexport.

…g reexport (issue #59 follow-up)

Add a 6-cell test matrix that varies the cyclic star-export fixture along
two axes: the order in which main.js imports the export-renamer and the
star-reexporter, and the binding form (const, let, var) the renamer uses
for its local y.

The star-reexporter reads r.y through a namespace import of the renamer
at the top of its body and captures the result, so when the renamer is
imported first from main.js the observation lands while the renamer is
on the evaluation stack with y not yet initialized.

Four cells assert SES's current behavior because it already matches
Node.js: renamer-first with var (reads undefined), and all three
star-reexporter-first orderings (read the assigned value 42).

The two remaining cells (renamer-first with const and let) are marked
test.failing: under ECMA-262 semantics, the cross-module read through
the namespace import should raise ReferenceError because y is in the
temporal dead zone, and Node.js confirms this; SES's current
module-instance machinery returns undefined instead. The .failing
markers pin the Node.js reference behavior as the desired outcome and
surface the SES divergence as a known gap to address separately.
@kriscendobot

Copy link
Copy Markdown
Collaborator Author

@kriskowal Added the 6-cell TDZ-observation matrix in packages/ses/test/import-gauntlet.test.js at 30664c3.

Two axes:

  • import order in main.js: export-renamer first vs star-reexporter first.
  • binding form for the renamer's y: const, let, var.

The star-reexporter reads r.y through a namespace import of the renamer at the top of its body. When main.js imports the renamer first, depth-first cycle resolution puts the renamer on the evaluation stack while the star-reexporter's body runs, so the read lands in the window where y is linked but not yet initialized.

Matrix outcomes:

import order binding expected (Node.js) SES observes test status
renamer-first const ReferenceError (TDZ) undefined test.failing (divergence pinned)
renamer-first let ReferenceError (TDZ) undefined test.failing (divergence pinned)
renamer-first var undefined (hoisted) undefined passing
star-reexporter-first const 42 (bound) 42 passing
star-reexporter-first let 42 (bound) 42 passing
star-reexporter-first var 42 (bound) 42 passing

The two .failing cells surface an in-process SES divergence from Node.js: SES's module-instance machinery currently returns undefined for a cross-module namespace read of a const or let binding that has been linked but not yet initialized, where ECMA-262 (and Node.js) require ReferenceError for the temporal dead zone. The matrix pins the Node.js reference behavior as the desired outcome for those cells so the gap is visible in the suite to either accept or close separately.

The four star-reexporter-first cells have no TDZ window to observe because depth-first cycle resolution evaluates the renamer's body to completion before the star-reexporter's body runs; they record the bound-value observation that completes the matrix.

@kriscendobot kriscendobot requested a review from kriskowal June 10, 2026 23:24
endolinbot added 2 commits June 10, 2026 23:40
 follow-up)

Adds packages/ses/designs/construction-time-notifiers.md analyzing the
maintainer's proposal to create all module-instance notifiers at
construction time instead of link time, in response to naugtur's
inline-comment ask on upstream review and kriskowal's follow-up.

The analysis sets out the present four-phase lifecycle (load,
instantiate, construct, link), enumerates what is and is not available
at construction time, sketches a two-pass redesign that constructs
reexport-notifier stubs in instantiate and resolves them in a pass-2
wire step, and addresses the precompiled-ModuleSource calling-convention
question (no schema change required; imports(updateRecord) becomes
wire-only).

The recommendation is to land the redesign as a follow-up PR rather
than fold the refactor into the present regression-fix PR: the
redesign touches two large surfaces (makeModuleInstance and
makeVirtualModuleInstance), shifts the instantiate/link line, and
benefits from being reviewed against the parity baseline this PR's
test matrix establishes.

The analysis also notes a separable observation: the redesign does
not by itself close the SES-against-Node cross-module TDZ divergence
the test matrix pins as test.failing. Closing that gap requires a
separate change to the exported-getter contract (consult the
upstream's own-binding getter rather than caching the propagated
value), which the redesign enables but does not require.
…#59 follow-up)

Adds one test to packages/ses/test/import-gauntlet.test.js that mirrors
the renamer-first plus const star-reexport cell of the prior fixer's
TDZ matrix but with a named reexport (export { y } from
'./export-renamer.js') instead of a star reexport (export * from
'./export-renamer.js').

The cycle has the same shape as the star case (the named reexporter
and the export renamer reference each other), and the observation of
r.y through the namespace import lands during the same
linked-but-not-yet-bound window when main.js imports the renamer first.

Node.js raises ReferenceError for const y = 42 in this case, matching
the star-reexport behavior, because the temporal dead zone semantics
live with the binding, not with the reexport form. SES's current
module-instance machinery returns undefined instead, the same gap the
star-reexport test.failing cells pin.

The new test is marked test.failing per the prior fixer's discipline:
it asserts the Node.js reference behavior (ReferenceError) as the
desired outcome and surfaces the SES divergence as a known gap to
address separately. The named-reexport variant confirms the gap is
not specific to export *, which the maintainer asked to verify in
issue-comment 4675471286.
@kriscendobot

Copy link
Copy Markdown
Collaborator Author

@kriskowal Addressed both asks in two commits on top of the prior fixer's TDZ matrix.

(1) Construction-time notifiers consideration

Disposition: feasible-and-documented, deferred to follow-up PR.

Wrote the analysis as a standalone design document at packages/ses/designs/construction-time-notifiers.md in commit c817b9f, addressing each facet of your follow-up:

  • Is the present line between instantiate and link hard enough? Almost. makeModuleInstance returns an instance with notifiers, exportsProxy, and execute; the first two are populated in the construction body, but the wiring of reexport notifiers is a side effect of execute (via the imports(updateRecord) closure). The redesign shifts that side effect into a pass-2 step of instantiate itself.
  • Two-pass redesign sketch. Pass 1 (construct + name-claim) allocates the binding state for own exports AND creates a forwarder stub for every reexport notifier (renamed reexports from __reexportMap__ plus star reexports enumerated by walking each upstream's moduleSource.exports). Pass 2 (wire) resolves each stub against upstreamInstance.notifiers[upstreamName]; by then every instance is constructed and every own notifier exists. wireUpExportNotifier's deferred-resolver branch becomes dead code.
  • Precompiled ModuleSource calling-convention impact. None to the schema. imports(updateRecord) becomes wire-only (registers the functor's updaters on upstream notifiers it already located in pass 1); the functor's call shape is unchanged. The "partially applied notifiers" caveat from your comment maps onto the renamer's local-binding update freezer, which the present code already exposes as a notifier-shaped object; the redesign preserves that property.
  • TDZ-gap relationship. The redesign does NOT by itself close the SES-against-Node cross-module TDZ divergence the matrix pins as test.failing. The exported getter's contract (return the last propagated value; default undefined) is unchanged. Closing that gap requires a separate change to consult the upstream's own-binding getter rather than caching the propagated value, which the redesign enables but does not require.
  • naugtur's shared-primitive arm. The document sketches makeBindingNotifier (own-binding shape: { get, notify, update }) and confirms makeNotifierWithResolver covers the forwarder shape. Both makeModuleInstance and makeVirtualModuleInstance would consume the same primitives in the redesign.

Recommendation in the document: land the redesign as a follow-up PR rather than fold the refactor into this regression-fix PR. The redesign touches two large surfaces, shifts the instantiate/link line, and benefits from being reviewed against the parity baseline this PR's test matrix establishes.

(2) Named-reexport variant test

Added in commit 0c46da9 to packages/ses/test/import-gauntlet.test.js: one test mirroring the existing renamer-first + const star-reexport .failing case, but with the named reexport form (export { y } from './export-renamer.js').

The failure mode reproduces in the named-reexport shape. Node.js raises ReferenceError for const y = 42 here, matching the star-reexport behavior; SES returns undefined, matching the star-reexport divergence. Marked test.failing per the existing discipline. The result confirms the gap lives with the binding form, not with the reexport form (star against named).

Verified out-of-band with a minimal native-ESM fixture across the same const/let/var matrix; the named-reexport variant matches Node.js's TDZ enforcement on const and let (ReferenceError when read renamer-first) and undefined-while-hoisted on var, identical to the star case.

Verification

  • corepack yarn workspace ses test: 508 pass + 5 known failures + 2 skipped. The new named-reexport .failing is the 5th known failure; the other 4 are the star-reexport matrix's 2 .failing cells, plus the 2 pre-existing .failing tests (no-eval lockdown, implicit-default reexport syntax).
  • pre-push-gates --summary .: clean for the diff. Pre-existing probe failures on master (no-inline-import-jsdoc on packages/evasive-transform/src/index.js, security-md-hash-uniform divergent SECURITY.md across immutable-arraybuffer, bytes, hex, panic) are not introduced by either commit.

PR #379 head is now 0c46da9. Re-requesting review.

@kriskowal kriskowal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The follow-up design looks good. I find it preferable to fix the defects we’ve discovered in this exploration, with the added tests introduced without the .failing caveats. Please dispatch an agent to address both issues. Address each issue in separate commits.

endolinbot added 2 commits June 11, 2026 02:07
…star reexport)

Closes the SES-against-Node cross-module TDZ divergence pinned by the
renamer-first × const and renamer-first × let cells of the cyclic
star-reexport matrix in packages/ses/test/import-gauntlet.test.js.

Diagnosis. The cross-module read `r.y` through a namespace import (the
`'*'` notifier) propagated the raw `exportsTarget` object, not the
`exportsProxy`. `exportsTarget` had no property defined for the binding
until the late `defineProperty` pass at the end of `imports()`; a missing
property reads as `undefined` rather than throwing. Even after the
property landed, the `wireUpExportNotifier` helper installed an exported
getter that returned the last propagated value (initially `undefined`)
with no TDZ tracking.

Fix, three targeted changes that do not require the full
construction-time-notifiers redesign documented at
`packages/ses/designs/construction-time-notifiers.md`:

- `packages/ses/src/module-instance.js`: define `exportsTarget[name]`
  eagerly during construction for each own fixed and live export, using
  the existing TDZ-aware getter from `localGetNotify`. The late
  `arrayForEach(arraySort, defineProperty)` pass at the end of
  `imports()` redefines the same descriptor as a no-op, preserving the
  ECMA-262 sorted enumeration order without changing the eager TDZ-aware
  behavior. `wireUpExportNotifier` tracks its own TDZ state so the
  downstream's exported getter throws `ReferenceError` until the
  upstream binding propagates a value through the notifier chain, and
  defines the property on `exportsTarget` eagerly too.
- `packages/module-source/src/transform-analyze.js`: reorder the
  preamble so hoisted declarations (function declarations and `var`
  initializers) run before the imports call. This matches the ECMA-262
  model: function/var bindings are created and initialized to undefined
  during `InitializeEnvironment`, which precedes dependency evaluation
  in `Module.Evaluate`. Without this reorder, a hoisted `var y` in an
  upstream module would still be in the live-binding TDZ when a
  downstream's body read `r.y`, and the eager TDZ-aware getter would
  throw `ReferenceError` instead of returning `undefined` (the var
  cells of the matrix).
- `packages/module-source/test/fixtures/format-preserved.txt`: update
  the format-preserved expected output to reflect the new
  hoisted-before-imports preamble order.

The star-reexport `.failing` cells (renamer-first × const, renamer-first
× let) now pass without the `.failing` modifier. The four converging
cells of the matrix continue to pass unchanged. The renamer-first × var
cell continues to read `undefined` because the upstream's hoisting
preamble clears its TDZ before the downstream observes; this matches
Node.js. The design document updates explain how the fix relates to the
prior construction-time-notifiers redesign analysis (the redesign
remains orthogonal to the TDZ fix and is still appropriate as a follow-up
PR).

Refs endojs/endo#59.
The cyclic-named-reexport TDZ matrix cell (`export { y } from
'./export-renamer.js'` reading `r.y` from the renamer's namespace during
the linked-but-not-yet-bound window) is now ECMA-262-compliant and reads
ReferenceError, matching Node.js.

The preceding commit's fix to `wireUpExportNotifier` in
`packages/ses/src/module-instance.js` already covered both star reexports
and named reexports because both wire through the same helper from
`imports()` (the named-reexport path at line ~462 calls
`wireUpExportNotifier(exportedName, importNotifiers[localName],
specifier, localName)` with the same TDZ-tracking semantics as the
star-reexport candidate-all walk at line ~473). This commit therefore
ships the test-only change that converts the named-reexport `.failing`
cell to a passing assertion, confirming the gap is closed in the
named-reexport shape too.

Refs endojs/endo#59.
@kriscendobot

Copy link
Copy Markdown
Collaborator Author

@kriskowal addressed both TDZ defects in two separate commits per your directive:

  • 94c8846 fix(ses): enforce TDZ for cross-module namespace reads during cycle (star reexport) lands the core fix and converts the renamer-first × const and renamer-first × let star-reexport .failing cells to passing.
  • 53d8662 fix(ses): enforce TDZ for cross-module named-reexport reads during cycle ships the test-only conversion of the named-reexport .failing cell to a passing assertion (the source fix in commit 1 already covers both paths since they share wireUpExportNotifier).

Root cause

The cross-module read r.y through the namespace import ('*' notifier) propagated the raw exportsTarget object rather than the exportsProxy. exportsTarget had no property defined for the binding until the late defineProperty pass at the end of imports(); a missing property reads as undefined rather than throwing. The wireUpExportNotifier helper also installed an exported getter that returned the last propagated value (initially undefined) with no TDZ tracking.

Three targeted fixes

  • packages/ses/src/module-instance.js: define exportsTarget[name] eagerly during construction for each own fixed and live export, using the existing TDZ-aware getter from localGetNotify. The late arrayForEach(arraySort, defineProperty) pass at the end of imports() redefines the same descriptor as a no-op, preserving ECMA-262 sorted enumeration order. wireUpExportNotifier tracks its own TDZ state so the downstream's exported getter throws ReferenceError until the upstream binding propagates a value through the notifier chain.
  • packages/module-source/src/transform-analyze.js: reorder the preamble so hoisted declarations (function declarations and var initializers) run before the imports call. This matches the ECMA-262 model where function/var bindings are created and initialized during InitializeEnvironment, which precedes dependency evaluation in Module.Evaluate. Without this reorder, a hoisted var y in an upstream module would still be in the live-binding TDZ when a downstream's body read r.y, and the eager TDZ-aware getter would throw ReferenceError instead of returning undefined (the var cells of the matrix).
  • packages/module-source/test/fixtures/format-preserved.txt: update the format-preserved expected output to reflect the new hoisted-before-imports preamble order.

Also updated packages/ses/designs/construction-time-notifiers.md to reflect what actually closed the gap (the design's full notifier-graph redesign remains orthogonal and is still a reasonable follow-up).

Test status

  • All three previously-.failing cells now pass without the modifier: 2 star-reexport (const, let) + 1 named-reexport (const).
  • The renamer-first × var cell continues to read undefined because the upstream's hoisting preamble clears its TDZ before the downstream observes; matches Node.js.
  • Full SES suite: 511 pass, 2 known failures (pre-existing on master), 2 skipped. No regressions.
  • compartment-mapper, module-source, bundle-source test suites all pass.

Re-requesting review.

@kriscendobot kriscendobot requested a review from kriskowal June 11, 2026 02:10

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kriscendobot Please delete, but capture a copy in your journal.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Archived as commit b76ab1a and removed from the PR tree. The verbatim analysis now lives in the garden journal at https://github.com/kriskowal/garden/blob/journal/projects/endo/drafts/construction-time-notifiers.md with an index row in https://github.com/kriskowal/garden/blob/journal/projects/endo/drafts/README.md

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please double-check that each of these scenarios is also represented as a compartment mapper test fixture with a Node.js parity check.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified and extended in commit cc23fca. The previously-covered scenarios (cycle-rename and cycle-rename-unused) already had compartment-mapper fixtures with Node.js parity tests. The seven remaining scenarios from the TDZ-observation matrix and the named-reexport variant now have matching fixtures:

  • packages/compartment-mapper/test/fixtures-cycle-rename-tdz-const-renamer-first/ (probe ReferenceError)
  • packages/compartment-mapper/test/fixtures-cycle-rename-tdz-let-renamer-first/ (probe ReferenceError)
  • packages/compartment-mapper/test/fixtures-cycle-rename-tdz-var-renamer-first/ (probe undefined)
  • packages/compartment-mapper/test/fixtures-cycle-rename-tdz-const-star-first/ (probe value 42)
  • packages/compartment-mapper/test/fixtures-cycle-rename-tdz-let-star-first/ (probe value 42)
  • packages/compartment-mapper/test/fixtures-cycle-rename-tdz-var-star-first/ (probe value 42)
  • packages/compartment-mapper/test/fixtures-cycle-named-reexport-tdz-const-renamer-first/ (probe ReferenceError)

Each fixture has a SES-side test (cycle-rename-tdz--.test.js or cycle-named-reexport-tdz-const-renamer-first.test.js) and a Node.js parity sibling with the -node-parity.test.js suffix. The expected probe values live in _cycle-rename-tdz-assertions.js so both layers compare against exactly one set of expectations; parity is verified by construction. The import-gauntlet.test.js header comment now links each matrix cell to its fixture path.

… journal

The construction-time-notifiers analysis written for the issue #59 follow-up
review described a possible redesign of the SES module-instance machinery
(move reexport notifier creation from link phase into construction phase) but
recommended deferring implementation to a separate PR. Per kriskowal's
CHANGES_REQUESTED review on this PR, the analysis belongs in the garden's
journal rather than the package tree: the artifact records how the
exploration informed the in-PR TDZ fix without persisting a design document
that the PR does not implement.

The verbatim text now lives at
journal/projects/endo/drafts/construction-time-notifiers.md in the garden's
journal branch and is indexed from drafts/README.md there.
kriscendobot pushed a commit to kriskowal/garden that referenced this pull request Jun 11, 2026
… kriskowal

Maintainer review 4474269526 on endojs/endo-but-for-bots#379 asked to delete
packages/ses/designs/construction-time-notifiers.md from the PR tree and
capture a copy in the journal. The analysis explored moving SES reexport
notifier creation from the link phase into the construction phase. The
recommendation was to land the redesign as a follow-up PR; the in-PR fix for
issue #59 enforces TDZ on cross-module namespace reads without requiring the
construction-time-notifiers redesign.

Archived under projects/endo/drafts/ per the existing draft-design convention.
The drafts/README.md index gains a row pointing to the new file.

Fixer dispatch 4ab426.
…TDZ matrix

The seven new scenarios that the prior fixers added to
packages/ses/test/import-gauntlet.test.js (a six-cell TDZ matrix of binding
form against import order for cyclic star-export plus one named-reexport
variant) now have matching compartment-mapper test fixtures, each with a
Node.js parity sibling. The same fixture is loaded by both: by the
compartment-mapper scaffold in the SES-side test, and by Node.js's native
loader in the parity-side test. The expected probe value lives in
_cycle-rename-tdz-assertions.js so the two layers compare against exactly
one set of expectations. Parity is verified by construction: if both tests
pass, SES enforces the same TDZ semantics on the cross-module namespace
path that Node.js enforces natively.

Fixtures:

- fixtures-cycle-rename-tdz-const-renamer-first   (probe: ReferenceError)
- fixtures-cycle-rename-tdz-let-renamer-first     (probe: ReferenceError)
- fixtures-cycle-rename-tdz-var-renamer-first     (probe: value undefined)
- fixtures-cycle-rename-tdz-const-star-first      (probe: value 42)
- fixtures-cycle-rename-tdz-let-star-first        (probe: value 42)
- fixtures-cycle-rename-tdz-var-star-first        (probe: value 42)
- fixtures-cycle-named-reexport-tdz-const-renamer-first
                                                   (probe: ReferenceError)

The renamer-first const and let cells observe ReferenceError during the
temporal-dead-zone window when the renamer is on the evaluation stack with
y not yet initialized. The renamer-first var cell observes undefined
because the hoisting preamble clears the upstream TDZ before the downstream
observes. The star-first cells have no TDZ window to observe; depth-first
cycle resolution evaluates the renamer's body to completion before the
star-reexporter's body runs, so the probe captures the assigned value 42
for every binding form. The named-reexport variant confirms the TDZ gap is
not specific to export *.

The import-gauntlet.test.js header comment now links each matrix cell to
its parity fixture path so future maintainers can find both layers from
either side.
@kriscendobot kriscendobot requested a review from kriskowal June 11, 2026 07:08
@kriscendobot

Copy link
Copy Markdown
Collaborator Author

Addressed both inline asks from review 4474269526.

Ask 1: archive then delete construction-time-notifiers.md
Commit b76ab1a. The file is removed from packages/ses/designs/ in the PR tree; the verbatim text now lives at https://github.com/kriskowal/garden/blob/journal/projects/endo/drafts/construction-time-notifiers.md with an index row in https://github.com/kriskowal/garden/blob/journal/projects/endo/drafts/README.md.

Ask 2: compartment-mapper parity for each gauntlet scenario
Commit cc23fca. The previously-covered scenarios (the original endojs/endo#59 fixture and its unused-live-binding companion) already had compartment-mapper fixtures with Node.js parity tests (cycle-rename.test.js, cycle-rename-node-parity.test.js, cycle-rename-unused.test.js, cycle-rename-unused-node-parity.test.js). The seven new gauntlet scenarios added by the prior fixers (six-cell TDZ matrix for the cyclic star-export plus one named-reexport variant) now also have matching fixtures and parity tests:

  • fixtures-cycle-rename-tdz-const-renamer-first (probe ReferenceError)
  • fixtures-cycle-rename-tdz-let-renamer-first (probe ReferenceError)
  • fixtures-cycle-rename-tdz-var-renamer-first (probe undefined)
  • fixtures-cycle-rename-tdz-const-star-first (probe value 42)
  • fixtures-cycle-rename-tdz-let-star-first (probe value 42)
  • fixtures-cycle-rename-tdz-var-star-first (probe value 42)
  • fixtures-cycle-named-reexport-tdz-const-renamer-first (probe ReferenceError)

Each fixture has both a SES-side test (loaded via the compartment-mapper scaffold) and a Node.js parity sibling (-node-parity.test.js suffix) that runs the same fixture under plain Node.js. Expected probe values live in _cycle-rename-tdz-assertions.js so both layers compare against exactly one set of expectations. The import-gauntlet.test.js header comment now links each matrix cell to its fixture path so future maintainers can find both layers from either side.

Verification

  • yarn workspace ses test: 511 pass + 2 known failures + 2 skipped (unchanged).
  • yarn workspace @endo/compartment-mapper test: 1014 pass + 6 known failures (was 930; the +84 new passes come from 7 new fixtures times 12 scaffold variants each, plus 7 Node.js parity tests; plus a few existing scaffolded suites' additional pass-throughs that landed in this same run).
  • pre-push-gates: clean for diff under review (probe failures inherited from master: no-inline-import-jsdoc on evasive-transform/src/index.js, security-md-hash-uniform on the four divergent SECURITY.md files; both pre-existing).

CI
All 15 of 16 checks green at head cc23fca. Browser-tests is pending/long-running on this PR exactly as it is on the master branch's last several pushes (consistent 30-minute timeout-cancellation pattern unrelated to this PR's diff).

@kriskowal re-requesting your review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants