run resume / fork: how to pick up post-compile config changes

[JuroOravec]

Problem

Run artifacts are self-sufficient by design — once a run is compiled, every subtask's noRunner, agentEnv, llmOverrides, agentProfileOptions, cedarScript, etc. is captured in the artifact. On saifctl run resume, the orchestrator reads seedSubtasks from the artifact verbatim (modes.ts:641-650); it does NOT re-read feature.yml / phase.yml / saifctl/config.ts.

This is the right default for resume integrity. But it leaves a gap:

I started a run. Subtask 0 failed for a config reason (e.g. the staging test runner errored because I forgot tests: { none: true }). I edited the config. Now what?

Today the only path is: start a fresh run. That re-compiles cleanly against the updated config, but discards everything the failed run accumulated — runCommits[], captured phase.baseRefs, any partial progress.

For my case it didn't matter (the run failed on subtask 0 with zero commits applied to the worktree). But the pattern generalises poorly: a run that completed steps 1–4 then died on step 5 because of a config typo in step 5's phase.yml can't recover. The user has to either re-run the whole feature from scratch or hand-edit the artifact JSON.

Why a naive --recompile flag is the wrong fix

The first instinct is saifctl run resume --recompile <runId> — re-read feature.yml + phase.yml, re-derive subtasks, splice into the artifact. This is tempting but creates several problems:

1. Run artifacts become tied to folder structure. Today an artifact is self-contained: you can move the feature dir, rename phases, even delete the feature entirely, and run resume still works (it doesn't re-touch the spec). --recompile makes resume depend on the on-disk state of saifctl/features/<feat>/ matching what the run was started with.

2. Phase set may have changed. If the user added new phases between start + resume, where do they slot in? After the current cursor? At the end? Skipped because they're "out of band"?

3. Phase set may have shrunk. If the user removed a phase that the run had already completed, do we delete its runCommits entries? Keep them but mark "orphan"? Fail loud?

4. Mid-stream changes need reconciliation logic. Run completed phases 1–4; phases 4–9 have been edited; phase 4 now has different agent.options / gate.script / testScope. Do we:

   • Discard phase 4's runCommits and re-run it? (Loses work.)
   • Keep phase 4's runCommits but apply the new config from phase 5 onward? (Inconsistent — phase 4 ran under old config.)
   • Ask the user interactively? (Adds an interactive step to a CLI that prides itself on being scriptable.)

The cost of getting this right is high, and the result is a brittle interaction that's hard to reason about across resume + fork + test modes.

Alternative: saifctl run edit <runId> (sketch)

Instead of re-deriving from disk, expose a surgical-edit command that operates on the artifact itself:

# Examples (shape TBD)
saifctl run edit <runId> --set 'subtasks[0].noRunner=true'
saifctl run edit <runId> --set 'subtasks[5].llmOverrides.globalModel=anthropic/claude-opus-4-7'
saifctl run edit <runId> --set-all 'subtasks[*].agentProfileOptions.effort=xhigh' \
  --where 'phaseId == \"01-foundations-impl\"'

The artifact stays self-sufficient. Edits are explicit and scoped (no "please figure out what changed since I last touched the dir" magic). Resume continues to be a faithful replay of what's currently in the artifact.

Open questions for the edit-command design:

• Schema validation on edit. Should --set re-run the per-phase validators (lockstep checks, security validators, etc.) against the edited subtask? Probably yes — otherwise the user can edit themselves into a state the compiler would reject.
• Editable surface. Not every subtask field should be editable mid-run. phaseId, criticPrompt, agentScript, gateScript content are foundational; noRunner, agentProfileOptions, llmOverrides, agentEnv are configuration. Need a documented allow-list.
• Path-language. subtasks[*].x.y with --where is shell-friendly. JMESPath-style. Worth evaluating.
• Atomicity. Multiple --set flags in one invocation should apply as one transaction. Halfway-edited artifacts are worse than failed edits.
• Audit trail. Each edit should leave a record on the artifact (edits[]: { timestamp, set, oldValue, newValue }) so resume can show "the artifact was edited at T1 to flip noRunner on subtask 5; this isn't a pristine compile".

Concrete trigger that motivated this

Detailed thread: https://github.com/safe-ai-factory/saifctl/issues/

My run `qr9faw7` failed on subtask 0 of the `workflow-api` feature because vitest exited with "No test files found" — my feature has no holdout tests (the test suite lives in the project codebase under src/specs/workflow/*.test.ts, gated by gate.sh → pnpm check:agent). The fix is tests: { none: true } in feature.yml — but it doesn't apply on resume.

A run edit qr9faw7 --set-all 'subtasks[*].noRunner=true' would have unblocked the resume cleanly without re-running anything that already worked.

Out of scope for this issue

• Re-shaping run resume's default behaviour. Default stays: faithful replay from the artifact.
• Mode 4 / cloud control-plane impact. Per-run edits in a fleet have a different shape; this issue is filesystem-CLI-mode-only.
• Test mode (saifctl run test). It already deviates from resume semantics; should be considered separately.

Decision needed

• Prefer run edit direction or some hybrid?
• If run edit: lock the editable-field allow-list before designing the path-language.
• Effort estimate: ~1 week for a v1 run edit covering noRunner + agentProfileOptions + agentEnv + llmOverrides; longer if we want full schema-re-validation on edit.