Repository Quality Improvement: MSTEST Diagnostic ID Scheme Governance

[Evangelink]

🎯 Repository Quality Improvement Report — MSTEST Diagnostic ID Scheme Governance

Analysis Date: 2026-06-09
Focus Area: MSTEST Diagnostic ID Scheme Governance
Strategy Type: Custom (repository-specific)

Executive Summary

A review of how MSTEST diagnostic IDs are allocated and documented across the codebase reveals several consistency gaps. The ID scheme covers two distinct ranges — analyzer rule IDs (MSTEST0001–0068, 0069 for source generation) and obsolete-API marker IDs (MSTEST0100–0106) — but there is no central documentation or registry for the 0100+ range, and a 30-ID gap (MSTEST0070–0099) is completely unallocated with no written explanation.

Three public API properties in TestResult (InnerResultsCount, DatarowIndex, ReturnValue) are tagged [Obsolete(..., error: true)] yet omit the DiagnosticId and UrlFormat parameters that every other MSTEST0100+ obsolete attribute provides. This breaks the uniform pattern that allows IDEs to surface documentation links and suppression pragmas.

Addressing these gaps improves the developer experience for consumers who encounter the obsolete APIs, makes the ID allocation auditable, and prevents accidental ID collisions as new diagnostics are added.

Full Analysis Report

Focus Area: MSTEST Diagnostic ID Scheme Governance

Current State Assessment

Metrics Collected:

Metric	Value	Status
Analyzer rule IDs (MSTEST0001–0068)	65 active + 4 reserved/removed	✅ Centrally tracked in DiagnosticIds.cs
Source-gen ID (MSTEST0069)	1	⚠️ Tracked only via a comment in DiagnosticIds.cs; constant lives in the other project
Obsolete-API IDs (MSTEST0100–0106)	7 in use	❌ Not in DiagnosticIds.cs; scattered across individual source files
Unallocated ID gap (MSTEST0070–0099)	30 IDs	⚠️ Completely empty; no documentation on intent
Obsolete error: true attrs missing DiagnosticId	3	❌ TestResult.InnerResultsCount, .DatarowIndex, .ReturnValue
Obsolete attrs with DiagnosticId + UrlFormat	14 (MSTEST0100–0106)	✅ Correct conditional-compilation pattern used

Findings

Strengths

• MSTEST0001–0068 are cleanly tracked in a single DiagnosticIds.cs file with reserved-ID tombstone comments.
• The multi-target conditional-compilation pattern (#if NET8_0_OR_GREATER && !DEBUG / #elif DEBUG / #elif NET8_0_OR_GREATER / #else) ensures older TFMs gracefully degrade when DiagnosticId/UrlFormat aren't available.
• UrlFormat = "(aka.ms/redacted) is consistently applied wherever DiagnosticId` appears in the 0100+ range.
• MSTEST0069 is clearly annotated in both its source file and DiagnosticIds.cs.

Areas for Improvement

• [HIGH] TestResult.InnerResultsCount, TestResult.DatarowIndex, TestResult.ReturnValue use error: true Obsolete but have no DiagnosticId or UrlFormat. Users receive a bare compile error with no IDE link and no way to add a suppression pragma by ID.
• [MEDIUM] MSTEST0100–0106 are not tracked in DiagnosticIds.cs. The only way to discover all allocated IDs is to grep the entire codebase.
• [MEDIUM] The 30-ID gap MSTEST0070–0099 has no written explanation. New contributors or reviewers cannot tell whether these IDs are intentionally reserved or accidentally skipped.
• [LOW] No developer-facing document (e.g., in docs/) explains the two-range scheme, making it unclear which range a future obsolete-API ID should use.

---

🤖 Suggested Improvement Tasks

Task 1: Add DiagnosticId and UrlFormat to TestResult's three error-level Obsolete properties

Priority: High
Estimated Effort: Small

TestResult.InnerResultsCount, TestResult.DatarowIndex, and TestResult.ReturnValue are all decorated with [Obsolete("This API is unused and has no effect.", error: true)] but lack the DiagnosticId and UrlFormat parameters present on all other MSTEST0100-series obsolete attributes.

Assign the next available IDs — MSTEST0107, MSTEST0108, MSTEST0109 — to these three properties respectively, applying the same conditional-compilation pattern used for MSTEST0100–0106 in Assert.cs, StringAssert.cs, and CollectionAssert.cs:

// Before (src/TestFramework/TestFramework/Attributes/TestMethod/TestResult.cs)
[Obsolete("This API is unused and has no effect.", error: true)]
[EditorBrowsable(EditorBrowsableState.Never)]
public int InnerResultsCount { get; set; }

// After
#if NET6_0_OR_GREATER && !DEBUG
[Obsolete("This API is unused and has no effect.", error: true,
    DiagnosticId = "MSTEST0107",
    UrlFormat = "(aka.ms/redacted)
#else
[Obsolete("This API is unused and has no effect.", error: true)]
#endif
[EditorBrowsable(EditorBrowsableState.Never)]
public int InnerResultsCount { get; set; }

Apply the same change to DatarowIndex (MSTEST0108) and ReturnValue (MSTEST0109). Also add a FrameworkConstants entry for the shared message string for consistency, and add a constant in the message resource if needed.

---

Task 2: Register MSTEST0100–0109 in DiagnosticIds.cs

Priority: Medium
Estimated Effort: Small

src/Analyzers/MSTest.Analyzers/Helpers/DiagnosticIds.cs is the central registry for MSTEST IDs used by the analyzer rules (0001–0068) but does not track the 0100+ range used for obsolete-API markers. This means the only way to audit all allocated IDs is to grep across multiple projects.

Add the 0100-series constants at the bottom of DiagnosticIds.cs, using the same naming and tombstone-comment style:

// Obsolete-API diagnostic IDs (MSTEST0100+): used with [Obsolete(DiagnosticId = ...)]
// to give IDE-navigable links to obsolete public APIs. These IDs are NOT Roslyn analyzer
// rules — they do not appear in the analyzers package's rule set.
public const string DoNotUseAssertEqualsRuleId                 = "MSTEST0100";
public const string DoNotUseAssertReferenceEqualsRuleId        = "MSTEST0101";
public const string DoNotUseStringAssertEqualsRuleId           = "MSTEST0102";
public const string DoNotUseStringAssertReferenceEqualsRuleId  = "MSTEST0103";
public const string DoNotUseCollectionAssertEqualsRuleId       = "MSTEST0104";
public const string DoNotUseCollectionAssertReferenceEqualsRuleId = "MSTEST0105";
public const string DoNotUseLegacyMSTestExecutorRunTestsRuleId = "MSTEST0106";
public const string DoNotUseTestResultInnerResultsCountRuleId  = "MSTEST0107";
public const string DoNotUseTestResultDatarowIndexRuleId       = "MSTEST0108";
public const string DoNotUseTestResultReturnValueRuleId        = "MSTEST0109";

---

Task 3: Document the MSTEST0070–0099 gap in DiagnosticIds.cs

Priority: Medium
Estimated Effort: Small

There are currently 30 unused IDs between the analyzer-rule range (MSTEST0001–0068) and the obsolete-API range (MSTEST0100+), with no explanation. When a new contributor adds the next analyzer rule, they need to know whether to use MSTEST0070, MSTEST0069 (already taken by source gen), or something else.

Add a block comment in DiagnosticIds.cs immediately after the last analyzer rule entry:

// MSTEST0069 is reserved — see comment above (owned by MSTest.SourceGeneration analyzer).

// MSTEST0070–MSTEST0099 are reserved for future analyzer rules.
// When adding a new analyzer rule, use the next sequential ID in the 0001–0068 range
// if one is still available, then continue from MSTEST0070.

// MSTEST0100+ are used for [Obsolete(DiagnosticId = ...)] on deprecated public API members
// (not Roslyn analyzer rules). See the "Obsolete-API diagnostic IDs" block below.

---

Task 4: Add a developer guide section for MSTEST diagnostic ID allocation

Priority: Low
Estimated Effort: Small

docs/AddingAnalyzerCodeFix.md walks contributors through adding a new analyzer rule but does not mention the ID numbering scheme. Add a short section explaining:

• The two-range structure (0001–0068/0069 for analyzer rules/source-gen, 0100+ for obsolete-API markers).
• How to pick the next available ID in each range.
• The multi-target conditional-compilation pattern required to use DiagnosticId on [Obsolete] in a multi-TFM library (showing the #if NET6_0_OR_GREATER && !DEBUG guard).
• That MSTEST0100+ IDs are tracked in DiagnosticIds.cs alongside analyzer IDs for discoverability.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type
2026-05-22	test-framework-api-ergonomics	Custom
2026-05-25	agentic-workflow-maintainability	Custom
2026-05-26	workflow-ecosystem-health	Custom
2026-05-27	test-diagnostic-experience	Custom
2026-06-08	todo-comment-policy-compliance	Custom
2026-06-09	mstest-diagnostic-id-governance	Custom

---

🎯 Recommendations

Immediate Actions (This Week)

1. Add DiagnosticId to TestResult Obsolete properties (Task 1) — Priority: High. Directly improves the experience for any consumer hitting these compile errors.

Short-term Actions (This Month)

2. Register MSTEST0100+ in DiagnosticIds.cs (Task 2) — Priority: Medium. Makes the full ID registry auditable in one place.
3. Document the MSTEST0070–0099 gap (Task 3) — Priority: Medium. Prevents accidental ID reuse by future contributors.
4. Developer guide for ID allocation (Task 4) — Priority: Low. Codifies the convention in the contributor documentation.

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-06-10 — Focus area selected based on diversity algorithm

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Repository Quality Improver workflow.{ai_credits_suffix} · [◷]( · ◷)

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main

• expires on Jun 11, 2026, 10:53 PM UTC