Document unreleased MTP telemetry items from main#54342
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the Microsoft.Testing.Platform (MTP) telemetry documentation to include additional telemetry data points observed in production telemetry output that aren’t yet represented in the article’s data point table, and marks those entries as coming from main (unreleased).
Changes:
- Adds new timing-related telemetry data points (adapter loading, host builder/creation, and request/load phases).
- Adds MSTest-related telemetry items (configuration/settings timeouts, configuration source, and usage payload summaries).
- Adds a reporter/app “graceful exit” signal telemetry item.
| | (main) | Boolean for capturing debug traces. | | ||
| | (main) | Timeout in milliseconds for class cleanup. | | ||
| | (main) | Timeout in milliseconds for class initialization. | | ||
| | (main) | Boolean for treating empty data sources as inconclusive. | | ||
| | (main) | Boolean for enabling cooperative cancellation. | |
There was a problem hiding this comment.
main is on purpose, this is ready for next release. Then it should be replaced with the actual version.
Evangelink
left a comment
Evangelink
left a comment
There was a problem hiding this comment.
Cross-checked the additions against TelemetryProperties.cs and MSTestTelemetryDataCollector.cs. Direction is good and most rows map cleanly. A few corrections and a few missing rows inline.
Other small things:
ms.date(02/25/2026) wasn't bumped — worth updating before merge.- Style mismatch with existing rows (existing uses
If …/ short noun phrases; these useBoolean for …/Timestamp when …). The new style is more informative because it discloses the value type, so I am fine keeping it — just calling out for awareness. - Splitting versioned vs. unversioned into two PRs is a nice call.
| | (main) | Timeout in milliseconds for class cleanup. | | ||
| | (main) | Timeout in milliseconds for class initialization. | | ||
| | (main) | Boolean for treating empty data sources as inconclusive. | | ||
| | (main) | Boolean for enabling cooperative cancellation. | |
There was a problem hiding this comment.
Minor — mstest.setting.cooperative_cancellation is AsTelemetryBool(settings.CooperativeCancellationTimeout). It reports whether a cooperative-cancellation timeout was configured, not whether the feature is "enabled" per se. Consider tweaking to e.g. "Boolean indicating if a cooperative cancellation timeout was configured." Not blocking.
| | (main) | Timeout in milliseconds for class initialization. | | ||
| | (main) | Boolean for treating empty data sources as inconclusive. | | ||
| | (main) | Boolean for enabling cooperative cancellation. | | ||
| | (main) | Source of MSTest configuration (runsettings or testconfig). | |
There was a problem hiding this comment.
The source code comment for mstest.config_source lists three possible values, not two: "runsettings, testconfig.json, or none". Suggest tightening to enumerate all three and wrap literals in inline code to match prevailing table style:
| | (main) | Source of MSTest configuration (runsettings or testconfig). | | |
| | (main) | Source of MSTest configuration (`runsettings`, `testconfig.json`, or `none`). | |
| | (main) | Boolean for mapping inconclusive tests to failed. | | ||
| | (main) | Boolean for mapping non-runnable tests to failed. | | ||
| | (main) | Boolean for ordering tests alphabetically by name. | | ||
| | (main) | Boolean indicating if test parallelization is enabled. | |
There was a problem hiding this comment.
MSTestTelemetryDataCollector.AddSettingsMetrics also emits two parallelization items that aren''t documented in this PR — should be added alongside this row if the intent is parity with production telemetry CSV:
mstest.setting.parallelization_scope— string (MethodLevel/ClassLevel).mstest.setting.parallelization_workers— number of parallel workers (only emitted when configured).
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Amaury Levé <amauryleve@microsoft.com>
…ckticks Changes: - Update ms.date to 06/10/2026 - Convert remaining 'Boolean for/indicating if' to 'If' style for consistency - Add backticks to config_source values (runsettings, testconfig.json, none) - Fix cooperative_cancellation description to reflect timeout configuration - Add missing custom_test_class_types row - Add missing parallelization_scope and parallelization_workers rows Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds telemetry items seen in production telemetry CSV that are not documented in the table yet, and marks them as (main) because they are not tied to a released MTP version in Changelog-Platform.md.
This includes:
This is intentionally split from the versioned PR so released-version rows stay explicit.