Skip to content

design-decision-gate: reduce max-turns 20→12, add explicit turn budget and stopping criteria#29168

Merged
pelikhan merged 3 commits into
mainfrom
copilot/deep-report-add-turn-budget-guardrail
Apr 29, 2026
Merged

design-decision-gate: reduce max-turns 20→12, add explicit turn budget and stopping criteria#29168
pelikhan merged 3 commits into
mainfrom
copilot/deep-report-add-turn-budget-guardrail

Conversation

Copilot AI commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

The Design Decision Gate was configured with max-turns: 20 but averaged only 7.1 turns — a 5× variance (4–20) that signals over-investigation on complex PRs with no hard stopping signal.

Changes

  • max-turns: 20max-turns: 12 — tightens the engine-enforced ceiling while preserving headroom for complex ADR generation scenarios

  • Turn Budget Allocation table — maps each step to a per-step turn cap (≤7 total), with turns 8–12 reserved for fallback, unexpected data gaps, or complex ADR generation:

    Step Max turns
    Read pre-fetched summary + gate check 1
    Fetch PR details (only if needed) 1
    ADR search (branch + PR body) 1
    Fetch linked issue ADR (only if referenced) 1
    Generate draft ADR or verify alignment 2
    Commit draft + post comment 1
    Total ≤ 7
  • Mandatory Stopping Criteria — six named exit conditions, each with a specific safe output to emit immediately:

    • Noop (neither condition met)
    • ADR found, no divergence → approved comment
    • ADR found, divergence → divergence comment
    • No ADR, decision inferable → push draft + blocking comment
    • No ADR, decision not inferable → comment with missing context
    • Turn 11 reached → emit best available output on turn 12 (hard engine limit), no new investigation

@github-actions

Copy link
Copy Markdown
Contributor

Hey @Copilot 👋 — great direction on adding a turn budget guardrail to the Design Decision Gate! Keeping agent runs bounded is an important reliability improvement.

A few things to address before this is ready for review:

  • No changes yet — the diff is currently empty. Once commits land, make sure the workflow files include the updated max-turns, prompt body changes, turn allocation table, and stopping criteria.
  • Description missing motivation — the PR body is a checklist of tasks, but doesn't explain why this guardrail is needed. A short paragraph summarising the motivation (e.g. prevent runaway agent loops, improve cost predictability) would help reviewers.
  • No tests — once the changes land, make sure make recompile and make agent-finish pass.

If you'd like a hand completing this, you can assign this prompt to your coding agent:

Complete the changes described in PR #29168 for the Design Decision Gate workflow:
1. Lower max-turns from 20 to 8 in the workflow frontmatter.
2. Update the Hard Turn Budget text in the prompt body to reflect 8 turns.
3. Add an explicit per-step turn allocation table to the prompt.
4. Add an explicit stopping criteria / decision tree section.
5. Run make recompile and make agent-finish to validate.
6. Update the PR body with a clear description of why this guardrail is needed.

Generated by Contribution Check · ● 1.2M ·

Copilot AI changed the title [WIP] Add turn budget guardrail to Design Decision Gate design-decision-gate: reduce max-turns 20→8, add explicit turn budget and stopping criteria Apr 29, 2026
Copilot AI requested a review from gh-aw-bot April 29, 2026 17:28
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot keep turn at 12

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/b393421d-1e34-44b7-95c5-0b5af1e7f3e4

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Apr 29, 2026

Copy link
Copy Markdown
Contributor Author

@copilot keep turn at 12

Done in f484b1fmax-turns is now 12. The prompt's hard budget, reserve-turn note, and hard-stop trigger (turn 11 → emit on 12) are all updated to match.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw _j9me15veeKU (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv (http block)
  • https://api.github.com/repos/github/gh-aw
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .default_branch (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title design-decision-gate: reduce max-turns 20→8, add explicit turn budget and stopping criteria design-decision-gate: reduce max-turns 20→12, add explicit turn budget and stopping criteria Apr 29, 2026
Copilot AI requested a review from pelikhan April 29, 2026 18:09
@github-actions github-actions Bot mentioned this pull request Apr 29, 2026
@pelikhan pelikhan marked this pull request as ready for review April 29, 2026 20:22
Copilot AI review requested due to automatic review settings April 29, 2026 20:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reduces the Design Decision Gate agent’s maximum investigation depth and adds explicit guidance for how to spend turns and when to stop, to prevent over-investigation on complex PRs.

Changes:

  • Lowered max-turns from 20 to 12 in the gate’s engine configuration and the generated lock workflow.
  • Added a per-step “Turn Budget Allocation” table to guide investigation sequencing.
  • Added explicit “Mandatory Stopping Criteria” to enforce early exits and a hard stop near the turn limit.
Show a summary per file
File Description
.github/workflows/design-decision-gate.md Updates engine turn limit and adds turn budgeting + mandatory stopping criteria guidance for the agent.
.github/workflows/design-decision-gate.lock.yml Regenerates the compiled workflow to reflect the new --max-turns 12 and related metadata.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 3

Comment on lines +120 to +122
| Read pre-fetched summary + gate check | 1 | `cat adr-prefetch-summary.json`; decide noop or proceed |
| Fetch PR details (only if needed) | 1 | `cat pr.json pr-files.json pr.diff` |
| ADR search (branch + PR body) | 1 | `find docs/adr`; inspect PR body links |
Comment on lines +134 to +139
- **Noop exit**: `has_implementation_label` is `false` AND `requires_adr_by_default_volume` is `false` → call `noop` and **stop**.
- **ADR found, no divergence**: ADR contains all four required sections and the diff does not contradict the decision → call `add-comment` (approved) and **stop**.
- **ADR found, divergence**: Divergences identified → call `add-comment` (divergence list) and **stop**.
- **No ADR, decision inferable**: Draft ADR generated → call `push-to-pull-request-branch` + `add-comment` and **stop**.
- **No ADR, decision not inferable**: Decision cannot be inferred from PR evidence → call `add-comment` explaining what context is missing and **stop**.
- **Turn 11 reached**: If you are on turn 11 and have not yet called a safe output, emit the best available output immediately on turn 12 — do not start a new investigation step. Turn 12 is the hard limit enforced by the engine.
- **ADR found, divergence**: Divergences identified → call `add-comment` (divergence list) and **stop**.
- **No ADR, decision inferable**: Draft ADR generated → call `push-to-pull-request-branch` + `add-comment` and **stop**.
- **No ADR, decision not inferable**: Decision cannot be inferred from PR evidence → call `add-comment` explaining what context is missing and **stop**.
- **Turn 11 reached**: If you are on turn 11 and have not yet called a safe output, emit the best available output immediately on turn 12 — do not start a new investigation step. Turn 12 is the hard limit enforced by the engine.
@pelikhan pelikhan merged commit cbd2abd into main Apr 29, 2026
39 checks passed
@pelikhan pelikhan deleted the copilot/deep-report-add-turn-budget-guardrail branch April 29, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[deep-report] Add Design Decision Gate turn-budget guardrail — 4–20 turn variance observed

4 participants