Effect Worker Compilation Architecture Plan

Status

Purpose: Replace the current manual worker_threads workspace-compilation experiment with an Effect Worker-managed design.
Initial runtime: Node.js.
Required target runtime: Browser/Web Worker.
Primary architectural constraint: The data owner remains the sole owner of the authoritative document store and symbol graph.
Failure policy: Avoid silent execution fallbacks. A configured topology either starts and operates successfully or reports an explicit failure.

Implementation progress

Goals

Use Effect Worker to create, manage, schedule, interrupt, and dispose compilation workers.
Achieve real CPU-parallel Apex parsing through multiple backing workers.
Eliminate the compilation-worker to coordinator to data-owner write-back path.
Keep compilation outside the data owner's serial write fiber.
Keep symbol graph mutations serialized and owned exclusively by the data owner.
Use the same compilation protocol and orchestration model in Node and browser runtimes.
Use a persistent, warm compilation pool for workspace and interactive compilation.
Bound memory use by committing results as compilation jobs complete.
Make initialization and runtime failures visible and deterministic.

Non-goals

Sharing mutable SymbolTable instances between workers.
Using SharedArrayBuffer in the first implementation.
Parallel mutation of the authoritative symbol graph.
Preserving the current manual worker_threads implementation as a runtime fallback.
Fully optimizing serialization before the architecture is validated.
Supporting dynamic worker-pool resizing in the first implementation.

Terminology

Worker count

compilation.poolSize is the number of backing compilation workers:

In Node.js, each Effect worker is backed by a Node worker_thread.
In the browser, each Effect worker is backed by a Web Worker.

At the architecture and configuration level, worker count and thread count represent the same scaling control. The existing WorkerConfig.poolSize setting already expresses this and should be used rather than introducing a second count setting.

Effect worker concurrency

Effect's pool concurrency controls how many requests may be leased concurrently to one backing worker. It does not create additional JavaScript threads. The existing compilation.concurrency setting maps directly to this Effect Worker pool option.

Apex parsing is synchronous CPU work. Consequently:

size: N, concurrency: 1

provides up to N CPU-parallel compilations, while:

size: 1, concurrency: N

still provides only one CPU execution thread.

The compilation pool uses:

{
  size: compilationPoolSize,
  concurrency: compilationConcurrency,
}

For synchronous CPU-bound parsing, poolSize determines CPU parallelism. Values of concurrency above 1 can admit and multiplex additional requests on a worker, but do not add parsing threads.

Data-owner request concurrency

Data-owner request concurrency controls how many requests its Effect worker can admit. It is distinct from compilation worker count and from the data owner's internal serial graph-write fiber.

Original Experiment Baseline

The current uncommitted experiment introduces WorkspaceBatchCompileOnDataOwner. The data-owner worker manually creates Node worker threads, sends source files to them, waits for every result, deserializes all results, merges them serially, and then terminates the threads.

This validates the broad idea of moving compilation orchestration closer to the data owner, but it has several structural issues:

Worker threads are created and destroyed for each request.
The threads are managed manually rather than through Effect Worker.
Compilation and thread startup occur inside dataOwnerWrite, monopolizing the serial write fiber.
All compilation results are retained before any merge occurs.
Compile requests are started with unbounded concurrency.
Manual round-robin assignment does not represent worker availability.
Worker failure can leave pending promises unresolved.
Interruption and failure do not guarantee worker cleanup.
The implementation is embedded in the general LSP worker entry point and exits that path using a thrown string.
The implementation depends directly on Node APIs.
threadCount, concurrency, and the existing compilation-pool settings describe overlapping or misleading concepts.
The direct path must explicitly preserve raw-document ingestion and version validation.

Target Architecture

LSP client
    |
    v
Coordinator / LCSAdapter
    |
    v
Data-owner worker
    |
    |-- authoritative document storage
    |-- authoritative public-API symbol graph
    |-- serial graph commit queue
    |
    `-- Effect SerializedWorkerPool
          |-- compiler worker 1
          |-- compiler worker 2
          |-- compiler worker 3
          `-- compiler worker N

The data owner owns the compilation pool's scope and orchestration. Effect owns the backing worker lifecycle and scheduling. Compilation workers parse source and return serialized results; they never mutate or directly access the authoritative graph.

Core Invariant

Compilation must not execute while holding the data-owner write fiber.

The required flow is:

receive compile request
    |
    |-- validate source/session/version
    |-- compile through Effect worker pool       outside write fiber
    |
    `-- dataOwnerWrite(commit result)             short critical section

The serial commit section should do only the following:

Revalidate the document version and workspace-load generation.
Deserialize or reconstitute the returned symbol table.
Call ApexSymbolManager.addSymbolTable.
Resolve the document's symbol-readiness latch.
Record the compilation outcome and metrics.

Compilation Worker Protocol

Request

Define a platform-neutral tagged request for one Apex file. Workspace batching belongs to the orchestration layer, not to the compiler worker.

class CompileApexFile extends Schema.TaggedRequest<CompileApexFile>()(
  'CompileApexFile',
  {
    payload: {
      uri: Schema.String,
      content: Schema.String,
      version: Schema.Number,
      detailLevel: DetailLevelSchema,
      collectReferences: Schema.Boolean,
      traceContext: Schema.optional(Schema.String),
    },
    success: SerializedCompilationResultSchema,
    failure: CompileApexFileErrorSchema,
  },
) {}

The exact Effect Schema syntax should follow the repository's installed Effect version and existing wire-schema conventions.

Result

Return a deliberately defined wire DTO rather than a SymbolTable class instance:

interface SerializedCompilationResult {
  uri: string;
  version: number;
  symbols: SerializedSymbol[];
  references: SerializedReference[];
  hierarchicalReferences: SerializedHierarchicalReference[];
  metadata: SerializedSymbolTableMetadata;
  parserDiagnostics: SerializedParserDiagnostic[];
  metrics: {
    compileMs: number;
    serializeMs: number;
    symbolCount: number;
    referenceCount: number;
    payloadSizeBytes: number;
  };
}

The first implementation may use structured cloning. Transferable buffers can be evaluated later if profiling shows serialization or copying to be material.

Failure model

Distinguish source diagnostics from infrastructure failures:

Syntax and semantic parser errors are successful compilation results containing diagnostics.
Worker startup failure, worker termination, schema failure, missing bundles, and unexpected compiler exceptions are typed infrastructure failures.
A stale document version is a typed rejected-commit outcome, not a compilation failure.

Compilation Worker Entrypoints

Extract the compilation logic into a shared handler with platform-specific runners:

compiler.worker.shared.ts
    |-- request schema
    |-- compilation handler
    `-- result serialization

compiler.worker.node.ts
    `-- NodeWorkerRunner + shared handler

compiler.worker.web.ts
    `-- BrowserWorkerRunner + shared handler

The shared handler must not import node:worker_threads, use workerData, inspect __dirname, or contain platform-specific startup behavior.

The dedicated entrypoints replace the current compileThreadMode branch and its thrown-string early exit.

Effect Worker Pool Ownership

The data-owner service bootstrap should allocate one scoped, persistent serialized worker pool:

Worker.makePoolSerialized<CompilationWorkerRequest>({
  size: compilationPoolSize,
  concurrency: compilationConcurrency,
  initialMessage: () =>
    new InitializeCompilationWorker({
      serverMode,
      logLevel,
      projectNamespace,
    }),
  onCreate: warmAndVerifyCompilationWorker,
});

The pool's scope must match the data-owner worker's lifetime. Effect is responsible for worker leasing, request scheduling, interruption propagation, and disposal.

onCreate should verify initialization and perform any required warmup before the language server reports the topology as ready.

Platform Layers

Keep pool orchestration platform-neutral and provide only the spawner layer differently:

CompilationPool service
    |
    |-- NodeCompilationWorkerLayer
    |     `-- Node worker_threads-backed Effect Worker
    |
    `-- BrowserCompilationWorkerLayer
          `-- Web Worker/MessagePort-backed Effect Worker

Node implementation

The first implementation may resolve the dedicated compiler-worker bundle using Node path facilities inside the Node platform layer. No Node import should escape that layer.

Browser implementation

The browser data-owner worker needs a resolvable compiler-worker URL. Extend worker initialization with an optional platform configuration value such as:

compilationWorkerUrl?: string;

The existing browser worker URL and Blob handling should be factored into a reusable spawner rather than reimplemented in compilation orchestration.

If nested Web Workers cannot be created in a supported browser host, initialization of the selected topology must fail explicitly. It must not silently switch to coordinator-local compilation.

Workspace-Load Orchestration

Workspace load should be modeled as an explicit session state machine:

Receiving
    -> Ingesting
    -> Compiling
    -> Committing
    -> Draining
    -> Complete

Any state
    -> Failed

Required sequence

Receive and validate all client batches.
Begin the data-owner workspace-load session.
Store raw documents in authoritative data-owner storage.
Submit individual public-API compilation requests with bounded concurrency.
Commit each result through dataOwnerWrite as soon as it completes.
Wait until every requested file has reached a terminal compile-and-commit outcome.
Drain deferred supertype resolution.
Mark the workspace loaded and publish completion.

The session must not report completion until both compilation and graph commit are complete for every accepted file.

Bounded result processing

Do not collect the whole workspace's serialized results before merging. A suitable initial shape is:

Effect.forEach(
  entries,
  (entry) =>
    Effect.gen(function* () {
      const result = yield* compilationPool.executeEffect(
        new CompileApexFile(toCompileRequest(entry)),
      );

      yield* dataOwnerWrite(commitCompilationResult(result));
    }),
  { concurrency: compilationPoolSize },
);

The implementation uses a bounded Effect Queue between compilation and commit. Its initial capacity is four completed results per compilation worker (eight with the default two-worker pool), which lets fast files enter the commit queue without retaining the whole workspace. Queue spans record high-water mark, full-buffer offer count and wait time, and empty-buffer take count and wait time so later tuning is driven by observed backpressure.

Completion-order commits are acceptable during workspace load because ordinary cross-file resolution is deferred. If tests expose an ordering dependency, add a small ordered-result buffer rather than retaining the entire workspace.

Interactive Compilation

After workspace-load parity is established, route interactive compilation through the same persistent pool:

didOpen and didChange compilation.
Request prerequisite compilation.
Public, protected, private, or full-detail enrichment as required.

The pool should accept individual file jobs with scheduling priority applied before submission:

Immediate: active editor compilation
High:      LSP request prerequisite
Low:       workspace preload

Workspace loading must not make every compilation worker unavailable to an active editor indefinitely. The first implementation may use the existing priority scheduler to control submission. A separate reserved interactive worker should be considered only if priority scheduling is insufficient.

Explicit Topology Selection

Replace runtime fallback chains with an explicit configuration mode:

type CompilationTopology =
  'disabled' | 'legacy-coordinator-pool' | 'data-owner-managed-pool';

During development, both implementations may coexist behind explicit selection. They must not automatically substitute for one another after initialization begins.

When data-owner-managed-pool is selected:

Pool creation failure fails worker-topology initialization.
Compiler-worker initialization failure fails worker-topology initialization.
Workspace compilation infrastructure failure fails the workspace-load session.
Failure is reported through the existing workspace-load failure channel.
The server does not silently compile on the coordinator or data-owner main thread.

A bounded retry of a pure compilation request after an actual worker termination may be added as an explicit retry policy. It is not a change of topology and must be observable through metrics. Start with no retry until failure behavior is tested.

Configuration

Use names that distinguish parallel workers from request concurrency:

{
  "workers": {
    "compilation": {
      "topology": "data-owner-managed-pool",
      "poolSize": 4,
      "concurrency": 1
    },
    "dataOwner": {
      "requestConcurrency": 10
    }
  }
}

compilation.poolSize and compilation.concurrency already exist in shared settings and the VS Code configuration. The data-owner-managed pool must consume both directly; documentation must make clear that only poolSize adds parsing threads.

Use 1 as the default concurrency because Apex compilation is synchronous CPU work. Keep the setting available as an Effect Worker admission/multiplexing control, and validate values above 1 with measurements rather than treating them as additional CPU parallelism.

Use 2 as the default compilation pool size to provide CPU-parallel parsing without making the initial per-workspace memory cost overly aggressive.

A reasonable initial automatic worker count is:

max(1, min(4, availableParallelism - 1))

The exact default should be validated against memory consumption because each worker loads the parser, compiler code, and supporting indexes. Browser defaults may need to be lower than Node defaults.

Versioning and Correctness

Every compilation request and result must carry:

URI.
Document version.
Workspace-load session or generation identifier where applicable.
Requested detail level.

The data owner must revalidate these values inside the serial commit section. A result must be rejected when:

A newer document version is authoritative.
The workspace-load session has been replaced or failed.
The requested detail level would regress an already committed table for the same version.

Interactive compilation should continue using symbol-readiness latches. A latch is armed when compilation is scheduled and resolved only after the corresponding graph commit succeeds or reaches a terminal rejected state.

Backpressure and Cancellation

Pool submission must be bounded by compilation.poolSize or another small multiple of it.
A cancelled or failed workspace-load session should interrupt outstanding compilation effects.
Results arriving after cancellation must fail generation validation and must not mutate the graph.
Data-owner shutdown must close the pool scope and terminate all backing workers.
Large source and result payloads must not accumulate in an unbounded queue.

Observability

Add spans or metrics for:

Pool creation and warmup.
Worker count and per-worker concurrency.
Queue wait time before worker acquisition.
Compile time.
Serialization time.
Result payload size.
Commit queue wait time.
Deserialization/reconstitution time.
Symbol graph merge time.
Version/session rejection count.
Worker crash and interruption count.
Workspace session totals and critical-path duration.

Avoid labeling the full UpdateSymbolSubset duration as transport overhead. It currently includes queue wait and data-owner processing. Measure transport, queue wait, and commit work separately before comparing architectures.

Implementation Phases

Phase 1: Protocol and shared compiler handler

Define CompileApexFile, initialization, success, and failure schemas.
Define the serialized symbol-table DTO explicitly.
Extract public-API compilation from worker.platform.shared.ts into a reusable handler.
Add unit tests for schema round trips and result reconstruction.

Exit criteria: A compilation request can be executed in-process against the shared handler and reconstructed into an equivalent SymbolTable.

Phase 2: Dedicated Node Effect Worker

Add a dedicated Node compiler-worker entrypoint.
Launch it with NodeWorkerRunner and the shared handler.
Add the compiler-worker bundle to the Apex LS build.
Add a Node integration test that starts one Effect worker and compiles a file.
Remove the compileThreadMode startup branch from the general worker entrypoint.

Exit criteria: One dedicated Node Effect worker compiles Apex and exits cleanly under scope finalization.

Phase 3: Persistent data-owner-managed pool

Add a compilation-pool service to data-owner bootstrap.
Create a fixed Effect serialized pool with size: compilation.poolSize and concurrency: compilation.concurrency.
Warm and verify all workers during topology initialization.
Ensure pool scope is tied to data-owner lifetime.
Add worker creation, interruption, crash, and shutdown tests.

Exit criteria: The data owner owns a persistent pool of N Effect workers, and profiling demonstrates up to N CPU-parallel compilations.

Phase 4: Workspace-load integration

Preserve raw-document ingestion as a required step.
Move compilation outside dataOwnerWrite.
Commit each completed result through a short dataOwnerWrite operation.
Add session and version validation at commit time.
Drain deferred supertype resolution only after all commits finish.
Report infrastructure failures without runtime topology fallback.
Remove WorkspaceBatchCompileOnDataOwner manual thread management.

Exit criteria: Workspace load completes through the Effect-managed pool with correct graph contents, bounded memory, and no manual worker-thread lifecycle code.

Phase 5: Browser implementation

Add the dedicated browser compiler-worker entrypoint.
Supply the browser Effect Worker spawner layer to the data owner.
Pass or derive a valid compiler-worker URL during worker initialization.
Validate Blob URL, CSP, nested-worker, and MessagePort behavior in the supported VS Code web host.
Run the same protocol and correctness suite in browser tests.

Exit criteria: The browser uses the same compilation protocol and data-owner orchestration without Node-specific imports or coordinator-local fallback.

Phase 6: Interactive compilation

Route didOpen, didChange, and didSave compilation through the same pool.
Preserve storage-before-compilation ordering.
Preserve readiness-latch behavior.
Prioritize active-editor work over background workspace work.
Remove the old top-level compilation worker after parity is demonstrated. Complete.

Exit criteria: Workspace and interactive compilation use one pool without regressing cold-open correctness or editor responsiveness.

Phase 7: Performance tuning

Compare worker counts against elapsed time and peak memory.
Measure compile, serialization, transfer, commit queue, and merge costs independently.
Evaluate transferable buffers only if result transport is material.
Tune worker-count defaults separately for Node and browser if necessary.
Consider retry policy only after worker-failure behavior is characterized.

Exit criteria: Defaults are supported by repeatable profiling rather than estimates.

Test Plan

Protocol tests

Request and result schema round trips.
Parser diagnostics survive serialization.
References, hierarchical references, metadata, and detail levels reconstruct correctly.
Typed infrastructure failures decode correctly.

Pool tests

Configured worker count is created.
concurrency: 1 prevents overlapping jobs within a single worker.
Multiple workers execute CPU work in parallel.
Scope closure terminates every worker.
Worker startup failure fails pool initialization.
Worker termination fails or interrupts the owning request without hanging.

Workspace-load correctness tests

Every ingested file reaches a terminal compile-and-commit outcome.
Public-API symbols are queryable after completion.
Same-file reference edges are present.
Deferred supertype edges are drained before completion.
Ordinary cross-file references remain available through on-demand resolution.
Stale and wrong-session results cannot mutate the graph.
Workspace failure cannot leave the data owner in an active load session.

Responsiveness tests

Data-owner reads proceed while background compilation is running.
An interactive compile can overtake queued workspace work.
Graph writes remain serialized.
One slow compile does not retain all other completed results in memory.

Cross-platform tests

Shared handler suite runs in Node and browser.
Node runner has no browser dependencies.
Browser runner has no Node dependencies.
Browser worker URL initialization succeeds in the supported VS Code web environment.

Performance Acceptance Criteria

Compare against the same workspace, machine, extension configuration, cache state, and open-editor state.

At minimum, capture:

Total workspace-load critical-path duration.
Files compiled per second.
Peak heap for the data owner and each compiler worker.
Compilation pool queue wait.
Mean and p95 compile time.
Mean and p95 commit queue wait.
Mean and p95 graph merge time.
Interactive compile latency during workspace load.
Worker initialization and warmup duration.
Failure and stale-result counts.

The experiment is successful when it demonstrates real multi-worker parsing, improves the workspace-load critical path, keeps interactive latency acceptable, and does not shift the dominant cost into serialization, memory pressure, or the serial graph commit queue.

Removal Checklist

After the new topology reaches parity:

Remove WorkspaceBatchCompileOnDataOwner.
Remove manual worker_threads creation and pending-promise bookkeeping.
Remove compileThreadMode and its thrown-string exit.
Remove threadCount terminology.
Remove silent workspace compilation fallback branches.
Remove the old top-level compilation worker topology after interactive compilation migrates.
Remove obsolete settings and compatibility aliases after the planned deprecation period.
Update topology documentation and diagrams.

Key Decision

The data owner owns compilation orchestration, Effect owns the compilation workers, and only graph commits—not compilation or worker waits—enter the data owner's serial write fiber.

Uh oh!

FilesExpand file tree

EFFECT_WORKER_COMPILATION_PLAN.md

Latest commit

History