Merge pull request #148 from eduralph/feat/wave-based-batch

feat(flow): process batches as dependency waves (integration stacking + opt-in merge)

Author: eduralph

docs/07-publish-and-act.md

@@ -37,6 +37,14 @@ ready — the harness never does. gramps' governance makes this explicit
 (INTEGRATION §10): *"Eduard opens fork PRs as draft, marks ready himself; builder
 commits and stops — no push/PR-open/ready-mark without explicit instruction."*
 
+This holds for a **batch**, too. A multi-id `pdca flow` runs the batch as dependency
+waves ([09 parallel lanes](../template/PCDA/quality-cycle/09-parallel-lanes.md)); in the
+default `stack` mode each wave's accepted work is folded onto a run-scoped integration
+branch the next wave builds on, and each dependent opens a **stacked** draft PR — the
+harness still never merges, so you review and merge the stack bottom-up yourself. Only the
+opt-in `[driver].wave_mode = "merge"` (own-repo / CD, where you hold merge rights on the
+base) relaxes this, `gh pr merge`-ing each wave before the next builds.
+
 In gramps' git history you can see cycles land this way — each result bundle gets
 a branch and a PR:
 

template/.claude/agents/planner.md.jinja

@@ -50,17 +50,19 @@ human confirms — quantity is theirs to decide, not yours.
 
 **Order the batch — set the scheduling fields (don't leave the order to chance).** When
 you brief several issues you are also deciding how they interleave: the flow runs the
-batch as a scheduled wave (docs 09). Before writing the briefs, map the batch's real
-shape — which issues build on another's change, and which touch the same files — then
-set, per brief, the three machine-parsed scheduling fields:
-
-- **`Depends on:`** — a genuine build-on dependency; the flow holds this bundle until
-  each prereq is COMPLETE.
-- **`Depends on (merged):`** — the stricter form: hold until the prereq's PR is *merged*;
-  use when this issue edits files a prereq also edits, so Do builds on the merged result
-  instead of colliding at merge time.
-- **`Conflicts with:`** — no dependency, but two issues edit a shared file, so they must
-  never run in the same concurrent wave.
+batch as an ordered sequence of dependency **waves** (docs 09). Each wave's bundles build
+in parallel; its accepted work is folded onto the base the next wave builds on, so a
+dependent completes in the same run. Before writing the briefs, map the batch's real shape
+— which issues build on another's change, and which touch the same files — then set, per
+brief, the two machine-parsed scheduling fields:
+
+- **`Depends on:`** — the PRIMARY field: a genuine build-on dependency. This bundle lands
+  in a LATER wave than each prereq and builds on its accepted result. (This subsumes the
+  old `Depends on (merged):` / `Stacks on:` — both are now just `Depends on`: the wave fold
+  gives the dependent the prereq's diff without waiting for a human merge. Don't reach for
+  them.)
+- **`Conflicts with:`** — no dependency, but two issues edit a shared file: they are
+  scheduled into DIFFERENT waves, never built blind on the same base.
 
 Set these from the batch's *real* dependency/conflict structure — an unordered batch
 either serialises needlessly or lets two bundles collide on a shared file and waste a Do.

template/PCDA/quality-cycle/09-parallel-lanes.md

@@ -52,16 +52,24 @@ The first defense against integration tangling is to not create it. Assign work
 
 Partitioning by issue id alone is not enough — it isolates the *runs* but not the *changes*. The information needed is already produced at Plan: root-cause analysis names the files / area a fix will touch. Lane assignment is therefore a **Plan-beat judgment** — the same place the human decides scope and which issues to brief ([03 - Cycle Automation](03-cycle-automation.md)) — not a mechanical sharding step. When the touched areas genuinely cannot be predicted, prefer fewer, broader lanes and lean on the integration check below.
 
-### Declared ordering — `Depends on:` / `Conflicts with:` [built]
+### Declared ordering — `Depends on:` / `Conflicts with:` → dependency waves [built]
 
-Manual wave-splitting (run a prerequisite batch to COMPLETE, *then* the next) enforces ordering by hand; it does not scale to a batch with a real dependency graph, which is exactly when the lane pool is most useful. A brief may instead **declare** its ordering constraints and let the scheduler enforce them:
+Manual wave-splitting (run a prerequisite batch to COMPLETE, *then* the next) enforces ordering by hand; it does not scale to a batch with a real dependency graph. A brief instead **declares** its ordering constraints and the scheduler computes the order: a batch handed to `pdca flow` runs as an ordered sequence of **dependency waves** (`waves.compute_waves`). Each wave holds only mutually-independent work; its bundles build in parallel, are signed off and published, and then the wave's accepted patches are **folded onto a run-scoped integration branch the next wave builds on** — so a dependent builds on its prerequisite's accepted result *within one run*, with no merge by the harness.
 
-- **`- **Depends on:** <id>[, <id>…]`** — a topological gate. The in-driver pool dispatches a bundle only once every declared prerequisite is **COMPLETE** (signed off, not merely built). Because a prereq reaches COMPLETE only after its sign-off in an earlier pass, a dependent waits across passes — exactly the manual wave plan, now machine-enforced.
-- **`- **Depends on (merged):** <id>[, <id>…]`** — a *stricter* gate for a dependent that **edits files a prerequisite also edits**. `Depends on` waits only for COMPLETE — which means "a draft PR was opened", **not merged**. A dependent's Do runs in a worktree off the target base (`origin/<base>`), so a prereq whose PR is still open is *absent* from that base: the dependent is built without the predecessor's diff and its PR conflicts at merge. This field holds the dependent until the prereq's PR is **merged into the base** (read from the prereq's recorded PR via `gh pr view`), so Do genuinely builds on the merged result. Because the flow only *opens* draft PRs (it never merges), a merge-gated dependent is **held across `pdca flow` runs**: its prereq is published in one run, a human merges that PR, and a later `pdca flow <dependent>` finds it merged and proceeds. Best-effort and fail-closed: anything not confirmable as merged keeps the dependent safely blocked.
-- **`- **Stacks on:** <id>[, <id>…]`** — auto-stacked chains (issue #123). Like `Depends on (merged)` it is for a dependent that **edits files a prerequisite also edits**, but instead of *waiting for the prereq to merge* (multi-run), it builds the dependent on the prereq's **just-produced branch** within the **same `flow` run** and publishes a **separate stacked PR** (`gh pr create --base <prereq-branch>`, one PR per item showing only that item's increment). A stacked dependent is eligible once its prereq is **COMPLETE with a published branch**; its Do worktree bases off `origin/<prereq-branch>` and its PR targets that branch — the base is **derived from the prereq's `publish.json`**, never written in the brief (it doesn't exist at Plan time). So a planned, file-overlapping refactor sequence (`203 → 207 → 204`) completes as a reviewable PR stack in **one invocation** — independents still run in parallel; a `Stacks on` chain is sequential within itself. **Merge the stack bottom-up with merge-commit / rebase-merge, not squash** (a squash drops the parent's commits, so a child retargeted to the base re-shows the parent's diff until rebased); publish warns if the target repo disallows merge commits. If a parent's branch later changes (e.g. a squash-merge between runs), rebuild the items stacked above it (`signoff --iterate-do`) on the new foundation. Use `Depends on (merged)` instead when you'd rather wait for each PR to merge before the next builds.
-- **`- **Conflicts with:** <id>[, <id>…]`** — a same-wave exclusion. Two bundles that touch a shared resource (e.g. both edit one `ci.yml`) are **never in flight in the same concurrent wave**; the pool serializes them across lanes while still parallelizing everything else.
+- **`- **Depends on:** <id>[, <id>…]`** — the topological edge. The dependent lands in a **later wave** than every prerequisite and builds on the integration branch that carries the prereqs' accepted diffs. Because the fold carries the diff forward without a merge, this single field now **subsumes** the former `Depends on (merged)` and `Stacks on`.
+- **`- **Conflicts with:** <id>[, <id>…]`** — an undirected "edit a shared resource" relation. Because each wave is folded onto the base before the next builds, two conflicting bundles must not share a wave; the leveler **orients each conflict pair into different waves** (by a deterministic id order, unless a dependency path already separates them), so the later one rebuilds on the earlier's folded result instead of colliding.
+- **`- **Depends on (merged):** / **Stacks on:**`** — *deprecated, still parsed.* Both fold into a plain `Depends on` edge: the wave model gives the dependent the prerequisite's accepted diff (via the integration branch) without the old cross-run merge wait (`Depends on (merged)`) or the single-chain branch-stacking (`Stacks on`, whose multi-parent gap the integration branch fixes). Author `Depends on`.
 
-The fields are **additive and backwards-compatible**: with none declared, every bundle is always eligible and dispatch is byte-for-byte the prior **sort-by-name pool**. An unschedulable graph — a cycle, or a dependency that is neither in the batch nor an already-COMPLETE bundle — is a **hard error rejected before any build** (`pdca flow` aborts up front). `pdca status` shows a `[blocked-by: <ids>]` flag so the queue reads as a DAG, not a flat list. Declared ordering complements lane planning: planning *avoids* integration tangling by code locality; `depends_on` / `conflicts_with` *enforce* the residual ordering that locality cannot express.
+With no fields declared the batch is a **single wave**, dispatched byte-for-byte the prior **sort-by-name pool**. An unschedulable graph — a cycle, or a dependency neither in the batch nor an already-COMPLETE bundle — is a **hard error rejected before any build** (`waves.compute_waves` raises; `pdca flow` aborts up front). `pdca status` shows a `[blocked-by: <ids>]` flag, and `pdca waves` prints the computed wave plan up front. A post-Do **overlap audit** flags two same-wave bundles whose patches touch a shared file but declared no `Conflicts with` — a likely planner omission.
+
+### Carrying a wave forward — stack (default) vs merge [built]
+
+How a wave's accepted work reaches the next wave's base is `[driver].wave_mode`:
+
+- **`stack`** (default) — fold the accepted patches onto a run-scoped integration branch on `origin` (push-only, so a **fork** contributor can do it) and open each dependent's PR as a **stacked PR** against it. The harness never merges; you merge the PR stack bottom-up with a **merge commit** (not squash, which would drop a parent's commits). A non-applying patch (an undeclared overlap) STOPs the run. STOP discipline holds throughout.
+- **`merge`** (own-repo / continuous-delivery only) — `gh pr merge` each non-final wave's PRs so the next wave builds on the genuinely-merged base. Needs **merge rights** on the base remote (a fork lacks them upstream — keep `stack` there) and relaxes STOP discipline; fail-closed (a non-mergeable PR STOPs the run).
+
+Optionally (`[driver].regate_between_waves`), the repo-scoped gates re-run over the folded integration tip before the next wave builds on it, so a combination that is red though each fix was green *alone* STOPs the run.
 
 ## Integration validation — at the merge boundary
 
@@ -78,7 +86,7 @@ The lanes parallelize the **unattended** band only. The three human touch points
 - **Check sign-off** is interactive — converge here. (A *single-workspace* run can batch sign-off across the fanned-out bundles into one cheap-first session; independent lane *copies* each carry their own sign-off queue, so the human attends them in turn — an ergonomic cost of full copies versus an in-driver fan-out.)
 - **Act** runs once, across the completed cycles of all lanes — serial by nature.
 
-So the shape is: **Plan (serial) → Do + Check fan out across lanes → sign-off (serial join) → publish → integration re-gate at the merge boundary → Act once.** Parallelism lives entirely in the unattended middle; planning and the merge re-gate carry correctness across the results.
+So the shape is, **per wave**: Plan (serial) → Do + Check fan out across lanes → sign-off (serial join) → publish → **fold onto the integration branch** the next wave builds on (or, opt-in `merge` mode, `gh pr merge`) → optional re-gate — repeated wave by wave, then **Act once** across the batch. Parallelism lives in the unattended middle of each wave; the wave ordering and the integration fold (then the human's bottom-up merge of the PR stack) carry correctness across the results.
 
 ## Two realizations — separate workspaces vs an in-driver pool
 

template/pdca.toml.jinja

@@ -35,6 +35,29 @@ lanes = 1
 # `$PDCA_WORKTREE` too (see the gate examples below). On by default; best-effort (a target
 # that isn't a worktree-capable git checkout falls back to in-place). Set false to disable.
 worktree = true
+# Wave-based batch sequencing (#wave-model). When `pdca flow` is given several ids it runs
+# them as an ordered sequence of dependency waves (computed from each brief's `Depends on`
+# / `Conflicts with`): a wave's bundles build in parallel, then its accepted work reaches
+# the NEXT wave's base one of two ways.
+#   wave_mode = "stack"  (default) — fold each wave's accepted patches onto a run-scoped
+#       integration branch the next wave builds on, and open each PR as a stacked PR. This
+#       is PUSH-ONLY (a fork has push), keeps the STOP discipline (the harness never merges;
+#       you merge the PR stack bottom-up with a merge commit, not squash), and completes a
+#       dependent batch in one run. The fork-safe default.
+#   wave_mode = "merge"  (own-repo / continuous-delivery ONLY) — `gh pr merge` each
+#       non-final wave's PRs so the next wave builds on the genuinely-merged base. Needs
+#       MERGE RIGHTS on the base remote (a fork contributor lacks them upstream — keep
+#       "stack" there) and relaxes the STOP discipline for the batch. Fail-closed: a
+#       non-mergeable PR stops the run.
+wave_mode = "stack"
+# The `gh pr merge` strategy when wave_mode = "merge": "merge" (a merge commit — auditable,
+# bisectable; the default), "squash", or "rebase".
+merge_method = "merge"
+# Optional integration re-gate (#wave-model). After each wave folds (stack mode), run the
+# repo-scoped gates over the folded integration tip before the next wave builds on it, so a
+# combination that is red though each fix was green ALONE stops the run. Off by default
+# (needs your repo-scoped gates); in merge mode the PR's own CI is the merge-boundary check.
+regate_between_waves = false
 # Act cadence (issue #109). Act is a cross-cycle beat that yields a real delta only once
 # enough cycles have frozen to show a pattern, so `pdca flow` auto-runs it only when this
 # many cycles have frozen SINCE the last Act review — counted across flow invocations

template/src/pdca_harness/cli.py

@@ -16,7 +16,7 @@
 from pathlib import Path
 
 from . import (act, brief, driver, flow, gates, merged, publish, queue, revalidate,
-               signoff, state)
+               signoff, state, waves)
 from .config import Config
 
 
@@ -80,6 +80,11 @@ def main(argv: list[str] | None = None) -> int:
     p_status = sub.add_parser("status", help="list bundle states (cheap-first queue)")
     p_status.add_argument("issue_id", nargs="?")
 
+    p_waves = sub.add_parser("waves",
+                             help="show the computed dependency-wave plan for a batch (no build)")
+    p_waves.add_argument("issue_ids", nargs="*",
+                         help="ids to schedule; none → every in-flight briefed bundle")
+
     sub.add_parser("queue", help="the cheap-first sign-off burn-down (AWAITING_SIGNOFF)")
 
     p_gates = sub.add_parser("gates", help="run the deterministic Check gates (driver + CI share this)")
@@ -151,6 +156,8 @@ def main(argv: list[str] | None = None) -> int:
         return _flow(cfg, args)
     if args.cmd == "status":
         return _status(cfg, args.issue_id)
+    if args.cmd == "waves":
+        return _waves(cfg, args.issue_ids)
     if args.cmd == "queue":
         return _queue(cfg)
     if args.cmd == "gates":
@@ -304,6 +311,34 @@ def _status(cfg: Config, issue_id: str | None) -> int:
     return 0
 
 
+def _waves(cfg: Config, ids: list[str]) -> int:
+    """Print the computed dependency-wave plan for a batch — deterministic, no build
+    (#wave-model). With no ids, schedules every in-flight briefed bundle. An unschedulable
+    graph (cycle / unresolved dep) is reported, not run."""
+    if ids:
+        bundles = [cfg.bundle(i) for i in ids if (cfg.bundle(i) / "brief.md").exists()]
+    elif cfg.bundle_root.exists():
+        bundles = sorted((d for d in cfg.bundle_root.glob("issue_*")
+                          if d.is_dir() and (d / "brief.md").exists()
+                          and state.state(d) not in (state.COMPLETE, state.DISCONTINUED)),
+                         key=lambda p: p.name)
+    else:
+        bundles = []
+    if not bundles:
+        print("(no briefed bundles to schedule)")
+        return 0
+    try:
+        plan = waves.compute_waves(cfg, bundles)
+    except ValueError as exc:
+        print(f"unschedulable: {exc}", file=sys.stderr)
+        return 1
+    print(f"{len(bundles)} bundle(s) → {len(plan)} wave(s) ({cfg.wave_mode} mode; "
+          f"each wave builds on the prior's accepted work):")
+    for k, wave in enumerate(plan):
+        print(f"  wave {k}: " + ", ".join(d.name.removeprefix("issue_") for d in wave))
+    return 0
+
+
 def _publish_flag(d: Path) -> str:
     """A COMPLETE bundle's publish state (#97): a real publish writes publish.json with the
     PR url; absent ⇒ accepted-but-unpublished (dry-run / no-target / failed / not-yet-run),
@@ -314,10 +349,16 @@ def _publish_flag(d: Path) -> str:
             return "  [close: no PR]"
         return "  [unpublished]"
     try:
-        url = json.loads(pj.read_text(encoding="utf-8")).get("pr_url")
+        rec = json.loads(pj.read_text(encoding="utf-8"))
     except (ValueError, OSError):
         return "  [published]"
-    return f"  [PR {url}]" if url else "  [published]"
+    url, base = rec.get("pr_url"), rec.get("base")
+    if not url:
+        return "  [published]"
+    # A stacked PR (#wave-model / #123) targets the wave integration branch, not the base —
+    # show ↑<base> so the human knows to merge the stack bottom-up.
+    stacked = rec.get("mode") in ("stacked-pr", "stacked")
+    return f"  [PR {url}{f' ↑{base}' if stacked else ''}]"
 
 
 def _blocked_by(cfg: Config, d: Path) -> list[str]: