Never classify a CLOSED issue as open-on-main in release-readiness (dotnet#36410)

<!-- Please let the below note in for people that find this PR -->
> [!NOTE]
> Are you waiting for the changes in this PR to be merged?
> It would be very helpful if you could [test the resulting
artifacts](https://github.com/dotnet/maui/wiki/Testing-PR-Builds) from
this PR and let us know in a comment if this change resolves your issue.
Thank you!

## Why
The release-readiness regression classifier could report a **CLOSED**
issue as an active `open-on-main` regression (Tier 2 — "wait for main
merge, then backport"). That's contradictory: an unmerged OPEN PR cannot
have closed a completed issue. It happens when a giant still-open
"Candidate" changelog PR `Fixes`-lists dozens of issues — its OPEN state
gets attributed to an already-completed issue. Real-world: issue dotnet#35615
(CLOSED/COMPLETED) surfaced under `open-on-main` in the SR9 tracker
while candidate PR dotnet#35716 was still open.

## What
`Classify-RegressionCandidate` now enforces: **never emit `open-on-main`
for a CLOSED issue.** When the aggregated best verdict is exactly
`open-on-main` and the issue is CLOSED:
1. First try the same comment-prose recovery the empty-candidate path
uses — a merged fix verifiably present on the SR wins →
`closed-fix-unlinked` (Tier 3, "no ship risk; add a closing reference
for traceability").
2. Otherwise fall to the honest `no-fix-yet` (Tier 3 for a CLOSED issue
via the existing `Get-OverallVerdict` downgrade) — the automation can't
pin a verified fix on this SR and the open candidate hasn't merged.

The shared recovery logic is extracted into a new
`Resolve-ClosedFixUnlinked` helper, called from both the
empty-`strongPrs` CLOSED path and the new guard, preserving the
fix-phrase gate, merged-on-SR gate, revert guards, and tooling-only
skips exactly.

**Strictly scoped:** only `open-on-main` + CLOSED is contradictory.
Every other verdict (`merged-on-main-no-backport`,
`backport-in-progress`, `rejected-from-sr`, `in-sr-*`,
`needs-human-review`) is unchanged even for CLOSED issues — those remain
legitimately actionable (the SR may still need the backport).
Genuinely-OPEN issues still get `open-on-main`.

## Tests
Three new unit tests in `Test-ReleaseReadiness.ps1`:
1. CLOSED issue + OPEN candidate on main + no comment fix →
`no-fix-yet`, proven non-blocking (🟢) via `Get-OverallVerdict`.
2. CLOSED issue + OPEN candidate + comment-cited merged fix on SR →
`closed-fix-unlinked` (recovery wins).
3. Regression guard: OPEN issue + OPEN candidate → stays `open-on-main`.

Full suite green (829/0).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: PureWeen <223556219+Copilot@users.noreply.github.com>

Author: PureWeen

.github/skills/release-readiness/scripts/Get-ReleaseReadiness.ps1

@@ -2033,6 +2033,93 @@ function Get-BackportPrsForSr {
     return @($list)
 }
 
+function Resolve-ClosedFixUnlinked {
+    <#
+    .SYNOPSIS
+        Recover 'closed-fix-unlinked' for a CLOSED regression issue whose fix
+        lives ONLY in comment prose (no closing keyword, no timeline link).
+
+    .DESCRIPTION
+        Maintainers routinely close a regression with a plain-text comment
+        ("fixed by PR #35028") that GitHub never turns into a structured link.
+        Recover the cited PR and — ONLY when it actually MERGED and its commit
+        is verifiably on THIS SR branch — classify 'closed-fix-unlinked': the
+        fix is present (no ship risk), but the issue<->PR link is missing and
+        should be added for traceability.
+
+        Two gates make prose evidence safe, and BOTH are required:
+          1. fix-phrase ONLY — the comment must pair the PR with fix/resolve/
+             close language ("was fixed by PR #X"). A BARE mention is rejected
+             because regression issues routinely name the CAUSE PR for context
+             ("Before PR #32080 ... After PR #32080 the behavior changed"), and
+             the cause naturally lives on the branch — it is not a fix.
+          2. merged AND on the SR branch — a cited PR that never merged, or
+             merged elsewhere, is not proof the fix shipped here.
+
+        Reverted fixes are dropped (a rolled-back fix is not a fix), mirroring
+        the $revertedPrSet / Revert-title handling on the SR-contents and
+        candidate paths.
+
+        Returns the 'closed-fix-unlinked' result hashtable, or $null when the
+        issue is not CLOSED or no cited PR survives both gates (caller falls
+        back to its own no-fix-yet handling).
+    #>
+    param($Ctx, $Issue, $RevertedPrSet)
+
+    $issueState = Get-AzdoProp $Issue 'state'
+    if ($issueState -ne 'CLOSED') { return $null }
+
+    $commentPrs = Get-IssueCommentPrs -Repo $Ctx.repo -IssueNumber $Issue.number
+    $verifiedFixes = @()
+    foreach ($cp in $commentPrs) {
+        # Gate 1: require explicit fix language. Bare mentions (the cause-PR
+        # blame pattern) are NOT fixes and must not reclassify the issue.
+        if ($cp.evidence -ne 'fix-phrase') { continue }
+        if ($cp.number -eq $Issue.number) { continue }   # self-reference
+        $info = Get-PrInfo -Repo $Ctx.repo -PrNumber $cp.number
+        if (-not $info) { continue }
+        if ($info.state -ne 'MERGED') { continue }
+        # A reverted fix is NOT a fix. Mirror the main SR-contents/candidate
+        # paths: drop PRs the SR later reverted, and drop PRs that are themselves
+        # rollbacks ("Revert ..." titles). Without this, the `(#<num>)` on-branch
+        # token checked below matches the reverted fix's number inside the revert
+        # commit's own subject `Revert "... (#num)" (#N)`, so a rolled-back fix
+        # would pass the on-branch gate and be reported as "No ship risk".
+        if ($RevertedPrSet.ContainsKey([int]$info.number)) { continue }
+        if (($info.title -match '(?i)^(?:\[[^\]]+\]\s+)?Revert\b') -or ($info.title -match '\[Revert\]')) { continue }
+        # Skip agent/skill/workflow PRs that only mention the issue for context.
+        if (Test-PrIsToolingOnly -Files $info.files) { continue }
+        $mergeSha = if ($info.mergeCommit) { $info.mergeCommit.oid } else { $null }
+        # Gate 2: presence on the SR branch via EITHER signal — direct SHA
+        # ancestry (fix merged straight to SR) OR the `(#<num>)` subject token
+        # (fix flowed in from inflight/main under a different SHA — the common
+        # case).
+        $onSr = (Test-CommitOnBranch -Sha $mergeSha -BranchRef "origin/$($Ctx.srBranch)") `
+                -or (Test-PrNumberOnBranch -PrNumber ([int]$info.number) -BranchRef "origin/$($Ctx.srBranch)")
+        if (-not $onSr) { continue }
+        $verifiedFixes += @{
+            number = [int]$info.number
+            title = $info.title
+            state = $info.state
+            mergeSha = $mergeSha
+            evidenceType = "comment-$($cp.evidence)"
+        }
+    }
+    if ($verifiedFixes.Count -gt 0) {
+        $prList = (@($verifiedFixes | ForEach-Object { "#$($_.number)" }) | Sort-Object -Unique) -join ', '
+        return @{
+            classification = 'closed-fix-unlinked'
+            confidence = 'high'
+            evidence = @("Issue is CLOSED and fix PR $prList is MERGED and present on $($Ctx.srBranch), but was never linked to the issue (no closing keyword, no timeline cross-reference). Linkage recovered from a closing comment that explicitly names the fix.")
+            candidateFixPrs = @($verifiedFixes | ForEach-Object {
+                @{ number = $_.number; title = $_.title; state = $_.state; evidenceType = $_.evidenceType }
+            })
+            recommendedAction = "No ship risk — fix is already in the SR. Add a closing reference for traceability (e.g. ``Fixes #$($Issue.number)`` in $prList, or link via the issue's Development panel) so future runs classify it automatically."
+        }
+    }
+    return $null
+}
+
 function Classify-RegressionCandidate {
     param($Issue, $CandidatePrs, $Ctx, $SrContents)
 
@@ -2171,74 +2258,10 @@ function Classify-RegressionCandidate {
 
         # ── FALLBACK: closed issue whose fix lives ONLY in comment prose ──
         # No timeline-cross-referenced candidate survived the evidence filter,
-        # but the issue is CLOSED. Maintainers routinely close a regression with
-        # a plain-text comment ("fixed by PR #35028") that GitHub never turns
-        # into a structured link. Recover the cited PR and — ONLY when it
-        # actually MERGED and its commit is verifiably on THIS SR branch —
-        # classify 'closed-fix-unlinked': the fix is present (no ship risk), but
-        # the issue<->PR link is missing and should be added for traceability.
-        #
-        # Two gates make prose evidence safe, and BOTH are required:
-        #   1. fix-phrase ONLY — the comment must pair the PR with fix/resolve/
-        #      close language ("was fixed by PR #X"). A BARE mention is rejected
-        #      because regression issues routinely name the CAUSE PR for context
-        #      ("Before PR #32080 ... After PR #32080 the behavior changed"), and
-        #      the cause naturally lives on the branch — it is not a fix.
-        #   2. merged AND on the SR branch — a cited PR that never merged, or
-        #      merged elsewhere, is not proof the fix shipped here.
-        $issueState = Get-AzdoProp $Issue 'state'
-        if ($issueState -eq 'CLOSED') {
-            $commentPrs = Get-IssueCommentPrs -Repo $Ctx.repo -IssueNumber $Issue.number
-            $verifiedFixes = @()
-            foreach ($cp in $commentPrs) {
-                # Gate 1: require explicit fix language. Bare mentions (the cause-PR
-                # blame pattern) are NOT fixes and must not reclassify the issue.
-                if ($cp.evidence -ne 'fix-phrase') { continue }
-                if ($cp.number -eq $Issue.number) { continue }   # self-reference
-                $info = Get-PrInfo -Repo $Ctx.repo -PrNumber $cp.number
-                if (-not $info) { continue }
-                if ($info.state -ne 'MERGED') { continue }
-                # A reverted fix is NOT a fix. Mirror the main SR-contents/candidate
-                # paths (see $revertedPrSet at the top of this function and the
-                # Revert-title skip in the candidate walk): drop PRs the SR later
-                # reverted, and drop PRs that are themselves rollbacks ("Revert ..."
-                # titles). Without this, the `(#<num>)` on-branch token checked below
-                # matches the reverted fix's number inside the revert commit's own
-                # subject `Revert "... (#num)" (#N)`, so a rolled-back fix would pass
-                # the on-branch gate and be reported as "No ship risk".
-                if ($revertedPrSet.ContainsKey([int]$info.number)) { continue }
-                if (($info.title -match '(?i)^(?:\[[^\]]+\]\s+)?Revert\b') -or ($info.title -match '\[Revert\]')) { continue }
-                # Skip agent/skill/workflow PRs that only mention the issue for context.
-                if (Test-PrIsToolingOnly -Files $info.files) { continue }
-                $mergeSha = if ($info.mergeCommit) { $info.mergeCommit.oid } else { $null }
-                # Gate 2: presence on the SR branch via EITHER signal — direct SHA
-                # ancestry (fix merged straight to SR) OR the `(#<num>)` subject
-                # token (fix flowed in from inflight/main under a different SHA —
-                # the common case).
-                $onSr = (Test-CommitOnBranch -Sha $mergeSha -BranchRef "origin/$($Ctx.srBranch)") `
-                        -or (Test-PrNumberOnBranch -PrNumber ([int]$info.number) -BranchRef "origin/$($Ctx.srBranch)")
-                if (-not $onSr) { continue }
-                $verifiedFixes += @{
-                    number = [int]$info.number
-                    title = $info.title
-                    state = $info.state
-                    mergeSha = $mergeSha
-                    evidenceType = "comment-$($cp.evidence)"
-                }
-            }
-            if ($verifiedFixes.Count -gt 0) {
-                $prList = (@($verifiedFixes | ForEach-Object { "#$($_.number)" }) | Sort-Object -Unique) -join ', '
-                return @{
-                    classification = 'closed-fix-unlinked'
-                    confidence = 'high'
-                    evidence = @("Issue is CLOSED and fix PR $prList is MERGED and present on $($Ctx.srBranch), but was never linked to the issue (no closing keyword, no timeline cross-reference). Linkage recovered from a closing comment that explicitly names the fix.")
-                    candidateFixPrs = @($verifiedFixes | ForEach-Object {
-                        @{ number = $_.number; title = $_.title; state = $_.state; evidenceType = $_.evidenceType }
-                    })
-                    recommendedAction = "No ship risk — fix is already in the SR. Add a closing reference for traceability (e.g. ``Fixes #$($Issue.number)`` in $prList, or link via the issue's Development panel) so future runs classify it automatically."
-                }
-            }
-        }
+        # but the issue is CLOSED. Recover a fix cited only in a closing comment
+        # (see Resolve-ClosedFixUnlinked for the fix-phrase + merged-on-SR gates).
+        $rec = Resolve-ClosedFixUnlinked -Ctx $Ctx -Issue $Issue -RevertedPrSet $revertedPrSet
+        if ($rec) { return $rec }
 
         return @{
             classification = 'no-fix-yet'
@@ -2335,6 +2358,39 @@ function Classify-RegressionCandidate {
     }
     $best = $perPrVerdicts | Sort-Object { $priority[$_.verdict] } | Select-Object -First 1
 
+    # ── CLOSED-issue guard against a contradictory 'open-on-main' ──
+    # 'open-on-main' means "the fix PR is still OPEN on main; wait for it to
+    # merge, then backport" — an ACTIVE (Tier-2) regression. That is impossible
+    # for a CLOSED issue: an unmerged PR cannot have closed it. This happens
+    # when a giant still-open 'Candidate' changelog PR `Fixes`-lists dozens of
+    # issues, so its OPEN state gets attributed to an already-completed issue
+    # (real-world: #35615 shown as open-on-main under SR9 while candidate #35716
+    # was still open). Never emit open-on-main for a CLOSED issue:
+    #   a. First try the same comment-prose recovery path 1 uses — a merged fix
+    #      verifiably on the SR wins → 'closed-fix-unlinked' (Tier 3).
+    #   b. Otherwise fall to the honest 'no-fix-yet' (Tier 3 for a CLOSED issue):
+    #      the automation can't pin a verified fix on this SR and the open
+    #      candidate hasn't merged. NOT an active SR regression.
+    # Scope is strict: ONLY open-on-main + CLOSED is contradictory. Every other
+    # verdict (merged-*, backport-in-progress, rejected-from-sr, in-sr-*,
+    # needs-human-review) stays as-is even for CLOSED issues — those are still
+    # actionable (the SR may still need the backport).
+    if ($best.verdict -eq 'open-on-main' -and (Get-AzdoProp $Issue 'state') -eq 'CLOSED') {
+        $rec = Resolve-ClosedFixUnlinked -Ctx $Ctx -Issue $Issue -RevertedPrSet $revertedPrSet
+        if ($rec) { return $rec }
+        return @{
+            classification = 'no-fix-yet'
+            confidence = 'medium'
+            evidence = @("Issue is CLOSED but the candidate fix PR (#$($best.pr.number)) is OPEN/unmerged on $($best.pr.baseRef) — an unmerged PR cannot have closed this issue; the real fix likely shipped elsewhere or the candidate is stale. Not an active SR regression.")
+            candidateFixPrs = @($strongPrs | ForEach-Object { @{
+                number = $_.number; title = $_.title; state = $_.state
+                baseRef = $_.baseRef; evidenceType = $_.evidenceType
+                onMain = $_.onMain; backports = $_.backports
+            } })
+            recommendedAction = 'Verify the fix is present on this SR (or add a closing reference); the open candidate PR has not merged.'
+        }
+    }
+
     $recAction = switch ($best.verdict) {
         'in-sr-active' { 'No action — fix is shipping' }
         'in-sr-reverted' { 'Investigate: backport landed and was reverted on SR' }

.github/skills/release-readiness/tests/Test-ReleaseReadiness.ps1

@@ -1771,6 +1771,99 @@ $clsRevertTitle = Classify-RegressionCandidate `
 Assert-Eq -Label "Comment-cited 'Revert ...' PR is a rollback, not a fix → stays no-fix-yet" `
     -Expected 'no-fix-yet' -Actual $clsRevertTitle.classification
 
+# ───── Classify-RegressionCandidate (CLOSED issue never open-on-main) ─────
+# Real-world case: SR9 tracker #35876 flagged CLOSED issue #35615 as `open-on-main`
+# (an ACTIVE Tier-2 regression) because a giant still-OPEN 'Candidate' changelog PR
+# (#35716) `Fixes`-listed dozens of issues. That is contradictory — an unmerged PR
+# cannot have closed a completed issue. The guard reroutes open-on-main + CLOSED to
+# a Tier-3 class: `closed-fix-unlinked` when a merged fix is verifiably on the SR, or
+# the honest `no-fix-yet` fallback otherwise. It must NOT change behavior for
+# genuinely-OPEN issues.
+Write-Host "`n[Unit] Classify-RegressionCandidate (CLOSED issue never open-on-main)" -ForegroundColor Cyan
+
+# A single OPEN 'Candidate' changelog PR on main that survives the evidence filter
+# (body `Fixes #35615` → closing-keyword; base=main; a real product file so it is
+# NOT tooling-only; not a Revert title) → the strong-PR walk verdict is open-on-main.
+function Get-BackportPrsForSr { param($Repo, $SrBranch, $SourcePrNumber) return @() }
+function Test-CommitOnBranch  { param([string]$Sha, [string]$BranchRef) return $false }
+function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return $false }
+function Get-PrInfo {
+    param($Repo, $PrNumber)
+    if ([int]$PrNumber -eq 35716) {
+        return [pscustomobject]@{
+            number      = 35716
+            title       = '[Candidate] SR9 changelog'
+            state       = 'OPEN'
+            baseRefName = 'main'
+            mergedAt    = $null
+            closedAt    = $null
+            body        = 'Fixes #35615'
+            mergeCommit = $null
+            files       = @([pscustomobject]@{ path = 'src/Controls/src/Core/Something.cs'; additions = 1; deletions = 0 })
+        }
+    }
+    # The comment-cited fix PR (used only in the recovery test below): MERGED into
+    # inflight/candidate, present on SR9 via the (#num) subject token.
+    return [pscustomobject]@{
+        number      = $PrNumber
+        title       = 'Fix the actual regression'
+        state       = 'MERGED'
+        baseRefName = 'inflight/candidate'
+        mergedAt    = '2026-06-01T00:00:00Z'
+        closedAt    = '2026-06-01T00:00:00Z'
+        body        = 'Fixes #35104'   # deliberately a DIFFERENT issue: recovery fires on the COMMENT citation, not this PR body
+        mergeCommit = [pscustomobject]@{ oid = 'c1d6d72768c0ffee' }
+        files       = @([pscustomobject]@{ path = 'src/Controls/src/Core/CollectionView.cs'; additions = 4; deletions = 0 })
+    }
+}
+
+# Test 1 — CLOSED issue + OPEN candidate on main + NO comment-cited fix →
+# reroute to `no-fix-yet` (NOT `open-on-main`).
+function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @() }
+$clsClosedNoFix = Classify-RegressionCandidate `
+    -Issue ([pscustomobject]@{ number = 35615; state = 'CLOSED' }) `
+    -CandidatePrs @(35716) `
+    -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr9'; mainBranch = 'main' } `
+    -SrContents @{ sourcePrs = @(); reverts = @() }
+Assert-Eq -Label "CLOSED issue + OPEN candidate PR → no-fix-yet, never open-on-main" `
+    -Expected 'no-fix-yet' -Actual $clsClosedNoFix.classification
+# `no-fix-yet` is raw Tier 1 via Get-VerdictTier, but Get-OverallVerdict downgrades a
+# CLOSED no-fix-yet to non-blocking (🟢). That downgrade is the whole point: it turns
+# the false blocking Tier-2 `open-on-main` into a Tier-3 (non-blocking) outcome.
+$vClosedNoFix = Get-OverallVerdict -Data @{
+    metadata = @{ mode = 'shipped' }
+    regressions = @(@{ classification = $clsClosedNoFix.classification; state = 'CLOSED' })
+    ci = @{ overall = 'green' }
+}
+Assert-Eq -Label "CLOSED no-fix-yet is non-blocking (Tier 3 effective → 🟢)" `
+    -Expected '🟢' -Actual $vClosedNoFix.symbol
+
+# Test 2 — CLOSED issue + same OPEN candidate on main, BUT a comment cites a MERGED
+# fix that is on the SR branch → the recovery wins → `closed-fix-unlinked`.
+function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @(@{ number = 35028; evidence = 'fix-phrase' }) }
+function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return ($PrNumber -eq 35028 -and $BranchRef -eq 'origin/release/10.0.1xx-sr9') }
+$clsClosedRecovered = Classify-RegressionCandidate `
+    -Issue ([pscustomobject]@{ number = 35615; state = 'CLOSED' }) `
+    -CandidatePrs @(35716) `
+    -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr9'; mainBranch = 'main' } `
+    -SrContents @{ sourcePrs = @(); reverts = @() }
+Assert-Eq -Label "CLOSED issue + OPEN candidate + comment-cited merged fix on SR → closed-fix-unlinked (recovery wins)" `
+    -Expected 'closed-fix-unlinked' -Actual $clsClosedRecovered.classification
+Assert-Eq -Label "closed-fix-unlinked recovery is Tier 3 (non-blocking)" `
+    -Expected 3 -Actual (Get-VerdictTier -Classification $clsClosedRecovered.classification)
+
+# Test 3 — REGRESSION GUARD: OPEN issue + OPEN candidate on main → the guard is
+# gated on CLOSED, so a genuinely-open regression STAYS `open-on-main`.
+function Get-IssueCommentPrs { param($Repo, $IssueNumber) return @() }
+function Test-PrNumberOnBranch { param([int]$PrNumber, [string]$BranchRef) return $false }
+$clsOpenIssue = Classify-RegressionCandidate `
+    -Issue ([pscustomobject]@{ number = 35615; state = 'OPEN' }) `
+    -CandidatePrs @(35716) `
+    -Ctx @{ repo = 'dotnet/maui'; srBranch = 'release/10.0.1xx-sr9'; mainBranch = 'main' } `
+    -SrContents @{ sourcePrs = @(); reverts = @() }
+Assert-Eq -Label "OPEN issue + OPEN candidate PR → stays open-on-main (guard is CLOSED-only)" `
+    -Expected 'open-on-main' -Actual $clsOpenIssue.classification
+
 # ───── Get-VerdictTier (deterministic tier table) ─────
 Write-Host "`n[Unit] Get-VerdictTier (deterministic tier table)" -ForegroundColor Cyan