Skip to content

fix: tolerate a concurrent symlink when linking the plugin dev SDK - #11059

Open
markkuhr wants to merge 3 commits into
paperclipai:masterfrom
markkuhr:fix/sdk-link-eexist-race
Open

fix: tolerate a concurrent symlink when linking the plugin dev SDK#11059
markkuhr wants to merge 3 commits into
paperclipai:masterfrom
markkuhr:fix/sdk-link-eexist-race

Conversation

@markkuhr

@markkuhr markkuhr commented Aug 7, 2026

Copy link
Copy Markdown

Thinking Path

  • Paperclip is the open source app people use to manage AI agents for work
  • The repo is a pnpm workspace. A postinstall step links the in-repo plugin SDK into each plugin package
  • linkSdkInto checks for an existing link, removes it, then creates its own
  • Workspace installs run that step for several packages at the same time, so two runs can interleave between the check and the create
  • The loser gets EEXIST, and one failed link fails the whole install
  • This pull request treats EEXIST as success, because the link already points where the run intended
  • The benefit is that pnpm install stops failing intermittently on a clean checkout

Linked Issues or Issue Description

No existing issue. Describing it here per CONTRIBUTING.md.

What happened?
pnpm install fails intermittently with EEXIST: file already exists, symlink ... @paperclipai/plugin-sdk. Re-running usually succeeds.

Expected behavior
The install completes. The symlink exists and points at the in-repo SDK.

Steps to reproduce
Run pnpm install on a clean checkout with several plugin packages present. The failure is timing dependent. The test added here reproduces it deterministically: twelve processes link the same package directory at a shared start instant, and eleven fail against the unfixed linker.

Paperclip version or commit
master, a388ea1.

Deployment mode
Not applicable. This is a build step.

Installation method
From source, pnpm install.

What Changed

  • linkSdkInto treats EEXIST from symlinkSync as success and returns true. Any other error still propagates. That is the whole behavior change: one error code on one call.
  • Added a concurrency regression test to the existing scripts/link-plugin-dev-sdk.test.js, which the test:release-registry job already runs. It drives separate processes, because in-process calls cannot interleave when the underlying filesystem calls are synchronous.
  • Hardened that test harness in a follow-up commit: handle a worker's error event so a spawn failure cannot leave the run hanging until the job timeout, sleep to just before the shared start instant instead of busy-waiting the whole window, import the module before the barrier so module load is not inside the race window, and report distinct worker errors instead of one identical line per worker.

Verification

node --test scripts/link-plugin-dev-sdk.test.js — 8 pass in 2.1s.

The contrast is the actual evidence. Restoring the original link-plugin-dev-sdk.mjs from master and re-running fails the new test on round 0 with 11 of 12 workers reporting EEXIST, reproduced 3 for 3.

On CI cost: the race test measured 2102ms on a GitHub-hosted runner (2224ms before the harness change) in the Typecheck + Release Registry job, which takes about 4 minutes overall. The harness change cut CPU from roughly 1074% to 107% at the same wall time, so the 60 spawned processes no longer pin every core while they wait. Worker and round counts are left at 12 and 5 — one round already fails 11 of 12 workers without the fix, and the extra rounds are cheap redundancy for machines where a given round happens to miss the window.

Risks

Low. The change narrows to one error code on one call. The postcondition is unchanged: after either path the link exists and points at the in-repo SDK. A real (non-symlink) install is still left alone, and a stale link pointing elsewhere is still replaced.

Two things worth naming for a reviewer:

  • The security-review check reports neutral, "Security Review Recommended". The flag is suspicious-test / shell-exec: the new test imports spawn from node:child_process. That is inherent to the test — the race window only opens under real process-level parallelism — and the spawned command is process.execPath running a literal source string with paths interpolated through JSON.stringify. No network, no shell, no external input.
  • Swallowing EEXIST assumes the racing creator is another run of this same script, which computes the same relative target for the same package directory. That holds here, because the target is derived only from the package directory and the repo root. A different writer creating something else at that path would be left in place, which matches what the existing "a real install has already populated it" branch does deliberately.

Model Used

Claude Opus 4.5 (claude-opus-4-5) for the fix and the initial test; Claude Opus 5 (claude-opus-5, 1M context) for the test-harness hardening and the CI cost analysis. Extended thinking, with tool use and code execution.

Checklist

  • I have included a thinking path that traces from project context to this change
  • I have specified the model used (with version and capability details)
  • I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
  • I have searched GitHub for duplicate or related PRs and linked them above
  • I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template
  • I have not referenced internal/instance-local Paperclip issues or links
  • My branch name describes the change and contains no internal ticket id
  • I have run tests locally and they pass
  • I have added or updated tests where applicable
  • I have updated relevant documentation to reflect my changes
  • I have considered and documented any risks above
  • All Paperclip CI gates are green
  • Greptile is 5/5 with no open P2s, recommendations, or follow-ups
  • I will address all Greptile and reviewer comments before requesting merge

Workspace installs run this for several packages at once. Each one checks for an
existing link, removes it, then creates its own, so two packages can interleave
between the check and the create and the loser fails the whole install with EEXIST.

Treat EEXIST as success. The link already points where this run intended, so the
postcondition holds and there is nothing to repair. Any other error still propagates.
@commitperclip

commitperclip Bot commented Aug 7, 2026

Copy link
Copy Markdown

✅ All checks passing — ready for Greptile review and maintainer approval.

— commitperclip

@greptile-apps

greptile-apps Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR makes plugin development SDK linking tolerate a concurrent EEXIST and adds a multi-process regression test for the race.

  • Handles EEXIST from the final symlink creation while preserving other errors.
  • Exercises concurrent linking across multiple workers and rounds.
  • Completes the requested PR description with rationale, verification, risks, model details, and checklist.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains, and the previously requested PR-description information has been supplied.

Important Files Changed

Filename Overview
scripts/link-plugin-dev-sdk.mjs Wraps SDK symlink creation so a concurrent EEXIST is treated as successful while other filesystem errors still propagate.
scripts/link-plugin-dev-sdk.test.js Adds a deterministic multi-process regression test covering concurrent SDK-link creation and child-process failures.

Reviews (3): Last reviewed commit: "test: harden the SDK link race test agai..." | Re-trigger Greptile

Comment thread scripts/link-plugin-dev-sdk.mjs
The window only opens under real parallelism, so the test drives separate
processes. In-process calls cannot interleave, because the underlying filesystem
calls are synchronous. Workers busy-wait to a shared start instant, then all link
the same package directory.

Reproduces reliably against the unfixed linker: eleven of twelve workers fail with
EEXIST on the first round.
Three problems with the harness, none with what it proves.

A worker that fails to spawn (EAGAIN under process pressure) emits "error" and
may never emit "close", so the promise never settled and the run would have hung
until the 20 minute job timeout. Handle "error" and report it as a failed worker.

Every worker busy-waited the full startup budget, pinning each core for the whole
window. Sleep to just before the shared start instant and spin only the last few
milliseconds. Same wall time, roughly a tenth of the CPU.

The dynamic import happened after the barrier, putting module load inside the
race window. Import first so every worker arrives at the symlink call warm, and
report distinct worker errors instead of one identical line per worker.

Reproduction is unchanged: 11 of 12 workers still fail on the first round against
the unfixed linker.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant