Embedding contract for claude-swarm

[simone-stacks]

Summary

claude-swarm assumes a person is driving: you launch it from the repository the agents should work on, watch the output, clean up afterwards. That works, and nothing here proposes changing it.

Driving it from another program: a CI job, a scheduler, anything that starts runs it doesn't watch and has to clean up after them. It is possible today, but only by depending on things that were never meant to be an interface: container name prefixes, /tmp/<project>-* paths reconstructed from the repo basename, a state file you have to source. It holds together until a rename in launch.sh silently breaks the integration, two runs share state that was never protected, or the code to inspect lives in a repository the engine checkout doesn't contain.

This issue lists the specific gaps and a set of changes that close them. Reference implementations exist as a stack of PRs, linked at the end.

Motivation

The problems

• Runtime state is predictable and shared. Everything lives under /tmp/<project>-*, keyed by the sanitized basename of the launch repo. The bare repo and submodule mirrors are chmod a+rwX (core.sharedRepository world), the state file is shell meant to be sourced, and nothing is locked: two checkouts with the same basename collide outright, and two start runs for the same project share one world-writable bare repo with no mutual exclusion.
• The engine has no target input. Agents always work on a clone of the launch repo. USAGE.md's answer for auditing another repository is passing TARGET_REPO/TARGET_REV through docker_args env passthrough: each container resolves the ref itself (a branch stays a branch, so agents in one run can land on different commits), clones over the network N times for an N-agent roster, and any credential needed to read a private target lives in every container's environment for the whole session.
• Submodules mirror top-level only. Mirrors are keyed by top-level submodule name and assume a $toplevel/.git/modules/<path> layout, so nested submodules fall back to network clones, and the assumption breaks under worktrees.
• Cleanup is destructive or manual. A stale container name is silently docker rm -f'd; a stale or unharvested bare repo is refused with an error message that tells you to rm -rf it. Both are fine when someone's watching; in an unattended loop they're where results get dropped without anyone noticing.
• There is no declared API. No control script, no versioned JSON, no container labels. An embedder greps docker ps for name prefixes and sources the state file and any rename inside launch.sh silently breaks that.
• post_process: null still builds a driver. A swarmfile without post_process appends the default driver to the SWARM_AGENTS build arg, so the image installs a CLI that nothing runs.

The changes

Five changes, independent enough to review separately, stacked in this order.

1. Private per-project runtime directory

Move the bare repo, submodule mirrors, lock, and state file under one 0700 directory per project, defaulting to $XDG_STATE_HOME/claude-swarm/<project>, with CLAUDE_SWARM_RUNTIME_DIR to override. The state file becomes validated, mode-600 JSON (claude-swarm.state/v1) instead of a sourced shell file.

Legacy /tmp state migrates itself on first run as a same-owner move. The move is validated first and fails closed on a held lock, symlink, foreign ownership, or destination collision, leaving the old paths untouched.

2. Host-resolved target mirrors

Make the target an explicit engine input. The host resolves TARGET_REPO@TARGET_REV once into a private mirror, fscks it, swaps it in transactionally, and mounts it read-only at /target-upstream. TARGET_REV is rewritten to the resolved 40-byte SHA before any container starts, so every agent works from the same immutable snapshot.

Submodule mirroring becomes recursive and manifest-driven: each initialized submodule gets a numbered mirror, and the host writes mirrors/manifest.tsv mapping display path to mirror. Containers walk the manifest, so nested .gitmodules are interpreted by the repo that owns them.

The read token is used only through a temporary host-side GIT_ASKPASS script and never enters a container. Agent passwordless sudo is revoked after the setup hook finishes.

docker_args can no longer set SWARM_READER_TOKEN, TARGET_REPO, TARGET_REV, TARGET_REV_BASE, TARGET_REV_BASE_REPO, SWARM_TARGET_MIRROR, or SWARM_TARGET_BASE_MIRROR; the engine strips them.

3. Transactional, fail-closed replacement

The bare repo is built aside, fscked, then swapped atomically with rollback. Replacement is refused entirely while any agent-work or swarm tip is not an ancestor of HEAD; stale-but-contained bares refresh in place. Bare permissions drop to owner-only.

Every agent and post-process container gets org.claude-swarm.{managed,project,engagement,role,agent-index} labels. An already-existing container name now fails the launch with a clear error instead of being force-removed. Labels are absent for containers created by older versions, so the code still falls back to the name prefix.

4. Versioned control plane

Add control.sh as the only programmatic surface embedders should touch:

• capabilities, paths, init, project-id, containers return versioned JSON (claude-swarm.control/v1, claude-swarm.containers/v1) and have no side effects.
• validate, start, stop, harvest, status, dashboard, post-process delegate to the engine.

paths reports runtime dir, bare repo, lock, state file, mirror dir, image name, and container prefix. containers reports status, exit code, OOM kill, engagement, role, and agent index per container. capabilities carries a feature list so future capabilities can be negotiated.

launch.sh validate runs the checks already done in cmd_start (prompt files exist, drivers exist, every profile resolves usable credentials) before the image build, so a missing key costs seconds instead of minutes.

Two one-line driver fixes ride along: kimi falls back to KIMI_MODEL_API_KEY, and a host ANTHROPIC_AUTH_TOKEN reports the token auth label.

5. Don't add a driver when post_process is absent

A swarmfile without post_process no longer appends the default driver to SWARM_AGENTS. Four lines plus a test fixture.

Breaking changes for embedders

1. Runtime paths move out of /tmp. Anything that read them should call control.sh paths instead.
2. The state file is JSON at mode 600; sourcing it no longer works.
3. docker_args cannot pass the target or reader-token variables anymore; set them in the host environment.
4. Agents lose passwordless sudo after the setup hook. Toolchain installs still run as root.
5. Every recursive submodule must be initialized at its pinned gitlink before start.
6. An existing container name fails the launch instead of being force-removed.
7. A bare repo holding an uncontained agent-work/swarm tip refuses replacement until harvested.
8. The bare repo is owner-only; core.sharedRepository world is gone.

Migration on the embedder side is small: drop the reserved docker_args passthroughs, clone from /target-upstream in the setup hook, call control.sh paths instead of reconstructing paths, and optionally lint against the reserved passthroughs.

Reference implementation

All five are implemented as a stack of PRs on a fork:

PR	branch	change
#2	feat/private-runtime	private per-project runtime, JSON state
#3	feat/host-target-mirrors	host-resolved target, recursive mirrors
#4	feat/bare-repo-lifecycle-guards	transactional replacement, container labels
#5	feat/control-plane	control.sh, launch.sh validate
#6	fix/post-process-null-agents	absent post_process adds no driver

The stack builds on the two open driver PRs (#113 Kimi, #114 Qwen), so those would need to land first. It can be opened as five PRs in that order, or folded into fewer if that reviews better.

One open question

Dropping core.sharedRepository world assumes the container's agent uid matches the host user's uid. That holds on a typical uid-1000 workstation and Docker Desktop remaps anyway, but on a Linux host with a different operator uid the agent would lose write access to /upstream. Keeping a slightly wider mode or pinning the uid at image build time are both fine alternatives; preference welcome before opening.

[moodmosaic]

Thanks for putting this together. 🙏

This is clearly need-driven, and I agree with the main point: claude-swarm assumes a person is operating it. Its current internals are not yet a safe interface for another program.

The /tmp paths, sourced state file, basename-derived identity, container-name discovery, and forced replacement are all worth fixing. A versioned control interface also makes sense for unattended use.

This is could end up being swarm-v1. 🚀

---

That said, after reading the actual diffs from your PRs (in your fork), I do not think the proposed implementation is ready yet.

The largest gap is locking. The patches expose an engagement.lock path, but no normal lifecycle operation acquires it. The only flock call is during legacy migration. Two concurrent starts can therefore pass preflight and race while replacing the bare repository and mirror tree.

A few other things need answers before this becomes a contract:

1. The default identity is still derived only from the repository basename. Two unrelated checkouts named project still share runtime, image, and container identity.

2. Making the bare repository and mirror tree owner-only breaks native Linux hosts where the host UID differs from the container’s agent UID.

3. Bare-repository replacement checks agent-work and swarm/*, but not agent-parked/*. Those refs are an intentional recovery path and may contain the only copy of an agent’s work.

4. stop leaves containers behind, while the next start refuses to continue if they exist. Without a retire or safe cleanup operation, an embedder must bypass the control interface to begin another run.

5. A later standalone post-process can resolve a moving target branch again. The state file records target_rev, but post-process does not load and reuse it. Every later phase should use the SHA resolved when the engagement started.

6. control.sh says its JSON commands are side-effect-free, but init creates directories and may migrate legacy state.

7. The engine never assigns or persists ENGAGEMENT_ID. The new labels therefore fall back to unknown or interactive unless the caller happens to provide it.

8. Automatic migration accepts the old world-writable bare repository when its top-level owner matches. That does not establish the provenance of its contents.

---

I also think this is too much to treat as one contract or one PR/feature pack. I would split it along the lifecycle invariants:

1. Private runtime state, unique identity, and real locking.

2. Safe replacement, retirement, cleanup, and preservation refs.

3. Host-resolved target snapshots.

4. Recursive submodule mirrors.

5. A versioned control interface.

The post_process driver fix in #6 is independent and could land on its own.

---

I also do not see a functional dependency on the Qwen or Kimi work. PR #5 happens to contain a small Kimi environment fallback, but that is unrelated to the control-plane work.

I would remove it from the list and base the resulting work directly on master.

---

I am happy to keep discussing the broader design. I agree strongly with the goal. I would first write down the lifecycle invariants, then build the interface around them. In particular, the engine needs to own locking, preservation refs, target identity, and cleanup policy.

Thanks again! The problem is worth solving. I just do not think the current implementation is ready to become the contract.