Enable human-gated release-readiness PR creation - #37693
Conversation
Keep readiness read-only by default while allowing explicitly authorized Arcade commands and fork-based manual backport PRs. Add contract tests for authorization, duplicate detection, main ancestry, and direct-release mutation safeguards. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1e7f77e3-4862-4b92-a662-d12b5918bb59
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 37693Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 37693" |
|
Azure Pipelines: 1 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Skill Validation Results
❌ Skill Validation Results —
|
There was a problem hiding this comment.
Pull request overview
This PR strengthens the separation between the deterministic release-readiness PowerShell engine (report-only) and the interactive release-readiness-agent by explicitly documenting a constrained “action mode” for SR backports that only activates after an explicit user request, and by adding contract tests to enforce that boundary in documentation.
Changes:
- Document an explicit, tightly-scoped SR backport execution path for the interactive
release-readiness-agent, while keeping deterministic scripts report-only. - Expand methodology guidance for automated vs manual (fork-based) backport paths, including “non-main selected candidate commit” exception rules under explicit release-captain authorization.
- Add unit/contract tests that assert the presence of required safety gates and prohibitions across the agent contract, skill docs, and methodology reference.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| .github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 | Adds contract assertions that documentation continues to encode the intended read-only default and constrained action gates. |
| .github/skills/release-readiness/SKILL.md | Clarifies deterministic vs interactive responsibilities; explicitly allows authorized SR backport follow-up without introducing write behavior into the skill scripts. |
| .github/skills/release-readiness/references/methodology.md | Documents the execution boundary, required pre-write checks, and the fork-only manual PR workflow (including non-main exception constraints). |
| .github/copilot-instructions.md | Updates repository-level agent/skill guidance so delegation and “explicit authorization” behavior is consistently documented. |
| .github/agents/release-readiness-agent.agent.md | Defines the “read-only by default; explicit backports only” contract, including explicit gates and hard prohibitions on release-ref mutation, merges, tags, builds, and scope creep. |
Generalize explicit release actions to fork-based PRs and add a fail-closed approval gate for the current PR head. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1e7f77e3-4862-4b92-a662-d12b5918bb59
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Suppressed comments (1)
.github/workflows/release-agent-human-approval.yml:108
- If the PowerShell policy throws before emitting JSON (e.g., invalid/missing input, download failure), $result will be empty/non-JSON. With
set -e, the subsequentjqcalls will fail and the step will exit due tojqinstead of the policy’s failure, which can mask the real error and prevent a useful step summary. Consider guarding thejqparsing (e.g.,if jq -e . "$result" >/dev/null 2>&1; then ... fi) and preservingexit "$status"as the only failure signal when JSON isn’t available.
result="$RUNNER_TEMP/human-approval.json"
set +e
pwsh -NoProfile -File "$POLICY_PATH" \
-ReviewsJsonPath "$REVIEWS_PATH" \
-ReviewerPermissionsJsonPath "$REVIEWER_PERMISSIONS_PATH" \
-PullRequestAuthor "$PULL_REQUEST_AUTHOR" \
-PullRequestHeadSha "$PULL_REQUEST_HEAD_SHA" \
-RequiredApprovals 2 > "$result"
status=$?
set -e
cat "$result"
{
echo "### Release-agent human approval"
echo
echo "- Required approvals: $(jq -r '.RequiredApprovals' "$result")"
echo "- Qualified approvers: $(jq -r 'if (.Approvers | length) == 0 then "_none_" else (.Approvers | join(", ")) end' "$result")"
} >> "$GITHUB_STEP_SUMMARY"
PureWeen
left a comment
There was a problem hiding this comment.
Automated review
❌ Policy Enforcement — The job applicability at .github/workflows/release-agent-human-approval.yml:20-23 is controlled only by the PR head-name convention or a literal, author-editable body marker. A release-agent action that creates a nonconforming branch, or a marker-only PR whose author later removes the marker, causes this required job to be skipped; skipped required jobs satisfy branch protection rather than enforcing the two-approval policy. Use a trusted, non-author-controlled provenance signal, or an always-running gate that fails closed when an intended release-agent PR is not positively identified. Flagged by: 2/3 reviewers.
.github/workflows/release-readiness.yml:40 adds approval-workflow changes to the PR-validation paths, but the same validation runs new assertions over .github/agents/release-readiness-agent.agent.md without including that agent contract in paths. A PR that weakens only the agent action boundary skips the workflow entirely, leaving those guards unexecuted. Add the agent file (or .github/agents/**) to the trigger. Flagged by: 3/3 reviewers.
.github/workflows/release-agent-human-approval.yml:57-59 runs under set -e; a deleted/renamed reviewer or transient API failure aborts the whole required gate even if two other current-head approvals qualify. The PowerShell policy already treats missing permission evidence as untrusted, so make each failed lookup record none and continue rather than turning an irrelevant reviewer into a merge-blocking outage. Flagged by: 3/3 reviewers.
Review method: 3 independent reviewers with adversarial consensus. The existing malformed-policy-output observation was not duplicated. The added approval-policy tests cover several decision-ordering cases, but the workflow trigger does not currently cover standalone agent-contract changes.
PureWeen
left a comment
There was a problem hiding this comment.
Adversarial review follow-up: independent reviewer assessment confirms that the three actionable issues from review 4988665802 remain. No additional analysis was performed.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d00747b7-96f3-4e7a-8dfb-e3a48db04b2d
|
@PureWeen addressed the latest feedback in 8c6e39c: the required job now always reports and preserves applicability through trusted |
PureWeen
left a comment
There was a problem hiding this comment.
Adversarial review
No actionable issues found at 8c6e39c8ed773772ec364944eba10dd4f302847e. Three independent reviewer assessments examined the trusted pull_request_target boundary, durable applicability tracking, current-head approval policy, error paths, documentation contract, and targeted regression coverage.
The only potential operational concern was independently re-evaluated and discarded: the bounded, sequential permission lookups intentionally fail closed and do not present a concrete PR-introduced regression.
Test coverage assessment: the focused approval-policy tests cover current-head binding, latest review decisions, author/bot exclusion, permissions, and malformed data; release-readiness validation now runs when the agent contract or approval workflow changes.
Method: 3 independent reviewers with adversarial consensus. This independent reviewer assessment posts a COMMENT-only verdict.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d00747b7-96f3-4e7a-8dfb-e3a48db04b2d
|
Merged current |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d00747b7-96f3-4e7a-8dfb-e3a48db04b2d
|
@copilot-pull-request-reviewer addressed the latest feedback in |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
.github/skills/release-readiness/tests/Test-ReleaseAgentHumanApproval.ps1:99
- This test uses a lowercased
mauibotlogin, which would not catch a case-sensitivity bug in the known-bot deny-list. Use a mixed-case login to ensure the policy treats known automation accounts case-insensitively.
$botApproval = Get-TestResult @(
(New-Review 4 'mauibot' 'APPROVED' '2026-01-01T00:03:00Z'),
$aliceApproval
)
Assert-Equal 'PAT-based bot account cannot satisfy the gate' $false $botApproval.Approved
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d00747b7-96f3-4e7a-8dfb-e3a48db04b2d
|
@copilot-pull-request-reviewer addressed the case-insensitive automation-account feedback in |
Note
Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!
Summary
release-agent/*fork branch and a durable PR-body marker; Arcadebackport/pr-*PRs are detected separatelyRelease Agent Human Approvalworkflow requiring two approvals on the current head SHA from distinct non-bot MAUI maintainers with write access, excluding the PR authorApproval enforcement
The workflow runs in a trusted context without checking out or executing pull-request code. It fully paginates review history, collapses each reviewer to their latest decision-bearing review, rejects stale-head approvals, filters GitHub Apps,
[bot]users and known PAT-based automation accounts, and verifies effective repository permission before counting an approval.After this PR merges, repository administrators must configure
Release Agent Human Approval / Require human approvalas a required status check on each protected target branch or ruleset. The workflow cannot make itself required through a repository-content PR, and it supplements rather than replaces the normal two-review policy.Validation
pwsh -NoProfile -File .github/skills/release-readiness/tests/Test-ReleaseAgentHumanApproval.ps1— 14 passedpwsh -NoProfile -File .github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1 -SkipE2E— 2,153 passedactionlint .github/workflows/release-agent-human-approval.ymlgit diff --check