[net11.0] Use scalable Helix job monitor uploader - #37865
Open
kubaflo wants to merge 5 commits into
Open
Conversation
Pin the post-Arcade dotnet#17331 monitor independently from the Helix submission SDK so result uploads use bounded parallelism, coordinated throttling, and resilient finalization. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
kubaflo
temporarily deployed
to
copilot-pat-pool
August 27, 2026 01:17 — with
GitHub Actions
Inactive
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 37865Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 37865" |
kubaflo
temporarily deployed
to
copilot-pat-pool
August 27, 2026 01:17 — with
GitHub Actions
Inactive
|
Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run. |
kubaflo
temporarily deployed
to
copilot-pat-pool
August 27, 2026 01:18 — with
GitHub Actions
Inactive
kubaflo
temporarily deployed
to
copilot-pat-pool
August 27, 2026 01:20 — with
GitHub Actions
Inactive
kubaflo
temporarily deployed
to
copilot-pat-pool
August 27, 2026 01:21 — with
GitHub Actions
Inactive
kubaflo
temporarily deployed
to
copilot-pat-pool
August 27, 2026 01:21 — with
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the MAUI Helix test stage template to pin the Helix Job Monitor package version independently from the Helix submission SDK, addressing scalability/reliability issues in Azure DevOps test-result uploads.
Changes:
- Add a new
helixJobMonitorVersiontemplate parameter (defaulting to11.0.0-beta.26425.2). - Stop deriving the monitor version from
eng/Versions.props; use the new parameter instead when installingMicrosoft.DotNet.Helix.JobMonitor. - Adjust the associated validation/error output for invalid monitor version values.
Report rejected values with Bash escaping so pipeline overrides are diagnosable without allowing raw control text into Azure logs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bind the Azure template parameter through the step environment so quotes and shell metacharacters remain data rather than template-expanded shell syntax. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use a default expansion under set -u so an absent environment binding reaches the explicit package-version validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The rewritten Job Monitor intentionally reports upload failures as warnings and exits successfully. Re-run untagged uploads within the shared timeout, fail closed when durable state is missing, and preserve real test failures until every upload is complete. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Author
|
/azp run |
|
Azure Pipelines: Successfully started running 3 pipeline(s). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!
Root cause
PR #37852 correctly isolated the Helix Job Monitor from MAUI's root tool manifest, but its exact-head
maui-prbuild 1569038 exposed a scalability problem in the branch-pinned monitor version:InProgressruns containing 90,650 uploaded result rows remained when the monitor timed out.11.0.0-beta.26379.102, which predates Arcade's transient-write retry fix and scalable result-processing rewrite.Fix
Pin the dedicated monitor job to
Microsoft.DotNet.Helix.JobMonitor11.0.0-beta.26425.2, independently from the Helix submission SDK.That package includes:
Arcade intentionally excludes upload failures from the monitor's exit code so an interrupted, untagged upload can be replayed by a later invocation. MAUI has only one monitor invocation, so the pipeline now re-invokes the pinned monitor when its final drain reports untagged jobs. Completed tagged uploads are skipped, incomplete jobs are replayed within the original shared timeout, and any real test failure is preserved until result publication is durable. Missing or changed durable-state output fails closed.
The Helix submission SDK remains unchanged at the branch-pinned version, so this only updates centralized result processing. The monitor package targets
net10.0withrollForward: Major, and runs under MAUI's pinned net11 runtime.No open MAUI PR addresses this follow-up. Increasing the six-hour timeout would only prolong the old uploader's blocked drain, while suppressing result publication would hide real test failures.
Validation
11.0.0-beta.26425.2from an empty NuGet cache.b49c2eb57bbccbbd4a16e535eb1edc2a9604d723.--max-wait-minutesand--fail-on-failed-tests.InProgress. The new replay guard classifies that exact raw summary as incomplete rather than green.