Skip to content

perf(codegen): migrate to incremental source generator#10035

Open
ReubenBond wants to merge 22 commits into
dotnet:mainfrom
ReubenBond:feature/incremental-source-generator
Open

perf(codegen): migrate to incremental source generator#10035
ReubenBond wants to merge 22 commits into
dotnet:mainfrom
ReubenBond:feature/incremental-source-generator

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Apr 23, 2026

Copy link
Copy Markdown
Member
Microsoft Reviewers: Open in CodeFlow

@ReubenBond ReubenBond force-pushed the feature/incremental-source-generator branch from 246b6f7 to e4e853f Compare April 28, 2026 01:47
ReubenBond and others added 6 commits April 28, 2026 07:36
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the custom equatable wrappers with direct ImmutableArray-based comparisons across the incremental generator models, and simplify extraction/preparation result handling while preserving diagnostics. Refresh the generator model equality and extractor tests to cover the new structural semantics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Stop copying analyzer-only project references into consumer outputs and cache metadata type resolution so repeated open-type lookups do not rescan assemblies during generation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reuse source generator services during proxy generation and improve incremental caching for serializer, metadata, and proxy outputs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restore transaction request services after deserialization and preserve method options needed by transaction invokers after copied or serialized delivery.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ReubenBond ReubenBond force-pushed the feature/incremental-source-generator branch from ea13b54 to f1efb2a Compare April 28, 2026 14:36
ReubenBond and others added 8 commits April 28, 2026 08:00
Split the remaining generator helpers into incremental-specific services and remove stale monolithic extraction shims.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move type manifest metadata output after generated serializers and proxies so generated source views list metadata last.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve the existing metadata generated source hint name and verify actual emission order separately so snapshots continue matching the monolithic generator output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve legacy generated source ordering in the incremental generator and normalize snapshot comparisons to the monolithic layout so existing verified output remains unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Register generated invokable activators from the emitted proxy artifact model so metadata matches generated output. Remove the transaction serialization workaround and add regression coverage for activator registration parity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restore several old-generator parity behaviors in the incremental generator: referenced assembly diagnostics, collision-safe hint names, partial declaration de-duplication, generated-code warning suppression, and stable incremental ordering.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Report diagnostics discovered while scanning referenced assemblies without source locations from those referenced compilations, since newer Roslyn rejects diagnostics whose locations are not part of the current compilation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This WIP pull request migrates Orleans’ code generation pipeline toward a Roslyn incremental source generator by introducing structurally comparable model types, refactoring generator components to use shared services/contexts, and adding characterization/unit tests to validate deterministic output and caching behavior.

Changes:

  • Refactors generator components (serializer/copier/proxy/invokable/metadata) to use new service/context types and shared utilities instead of the legacy CodeGenerator.
  • Adds incremental-friendly model types (*Model, TypeRef, TypeMetadataIdentity, StructuralEquality, etc) plus new generator context types.
  • Adds a large suite of tests covering ordering stability, caching, hint-name collisions, referenced-assembly behaviors, and diagnostic parity.
Show a summary per file
File Description
test/Orleans.CodeGenerator.Tests/TestCompilationHelper.cs New shared Roslyn compilation helper for generator tests.
test/Orleans.CodeGenerator.Tests/ReferencedAssemblyDiagnosticParityTests.cs New tests asserting diagnostic parity for referenced-assembly scenarios.
test/Orleans.CodeGenerator.Tests/PartialDeclarationDeduplicationTests.cs New tests ensuring partial declarations are deduplicated in outputs/diagnostics.
test/Orleans.CodeGenerator.Tests/OrleansSourceGeneratorTests.cs Updates existing generator tests for multiple generated outputs and incremental generator plumbing.
test/Orleans.CodeGenerator.Tests/ModelExtractorTests.cs New tests validating extracted incremental models and identity stability.
test/Orleans.CodeGenerator.Tests/IncrementalOrderingStabilityTests.cs New characterization tests for deterministic output under syntax tree reordering.
test/Orleans.CodeGenerator.Tests/IncrementalModelEqualityTests.cs New tests verifying structural equality semantics used by incremental caching.
test/Orleans.CodeGenerator.Tests/IncrementalCachingTests.cs New tests verifying incremental pipeline caching behavior via tracked steps.
test/Orleans.CodeGenerator.Tests/HintNameCollisionTests.cs New tests ensuring hint-name collision handling emits all expected sources.
test/Orleans.CodeGenerator.Tests/GeneratedWarningSuppressionTests.cs New tests ensuring generated sources don’t trigger XML doc diagnostics.
test/Orleans.CodeGenerator.Tests/DiagnosticTests.cs New diagnostic-focused tests for invalid patterns and generator resilience.
src/api/Orleans.Transactions/Orleans.Transactions.cs Formatting-only change in generated API surface file.
src/Orleans.CodeGenerator/SyntaxGeneration/FSharpUtils.cs Switches ID extraction utility usage for F# record member handling.
src/Orleans.CodeGenerator/SerializerGenerator.cs Refactors serializer generation to use IGeneratorServices + shared utilities.
src/Orleans.CodeGenerator/ProxyGenerator.cs Refactors proxy generation to use IGeneratorServices/CopierGenerator and shared utilities.
src/Orleans.CodeGenerator/ProxyGenerationContext.cs Introduces a dedicated context for proxy/invokable generation and emission.
src/Orleans.CodeGenerator/Properties/IsExternalInit.cs Adds IsExternalInit shim for netstandard builds.
src/Orleans.CodeGenerator/Model/TypeRef.cs Adds string-based type reference for incremental model storage.
src/Orleans.CodeGenerator/Model/TypeParameterModel.cs Adds type parameter model for incremental model representation.
src/Orleans.CodeGenerator/Model/TypeMetadataIdentity.cs Adds assembly+metadata-name identity for cross-assembly disambiguation.
src/Orleans.CodeGenerator/Model/StructuralEquality.cs Adds helpers/comparers for stable structural equality/hash behavior.
src/Orleans.CodeGenerator/Model/SourceLocationModel.cs Adds source-location model used for deterministic ordering.
src/Orleans.CodeGenerator/Model/SerializableTypeModel.cs Adds serializable type incremental model with structural equality.
src/Orleans.CodeGenerator/Model/SerializableTypeDescription.cs Adjusts description surface to align with new generation flow/utilities.
src/Orleans.CodeGenerator/Model/RegisteredCodecModel.cs Adds model for manually registered codecs/activators/copiers/converters.
src/Orleans.CodeGenerator/Model/ReferenceAssemblyModel.cs Adds referenced-assembly aggregate model for incremental compilation-based extraction.
src/Orleans.CodeGenerator/Model/ProxyOutputModel.cs Adds model for per-interface proxy output ownership (invokables/activators).
src/Orleans.CodeGenerator/Model/ProxyMethodDescription.cs Refactors to use ProxyGenerationContext and updated method-parameter field handling.
src/Orleans.CodeGenerator/Model/ProxyInterfaceModel.cs Adds incremental proxy interface model (proxy base + methods + identity).
src/Orleans.CodeGenerator/Model/ProxyInterfaceDescription.cs Refactors description to use ProxyGenerationContext and shared utilities.
src/Orleans.CodeGenerator/Model/MethodModel.cs Adds incremental method model used for invokable/proxy generation.
src/Orleans.CodeGenerator/Model/MetadataModel.cs Trims legacy metadata state; retains proxy-base invokable base type mapping.
src/Orleans.CodeGenerator/Model/MetadataAggregateModel.cs Adds aggregate model to drive final metadata emission deterministically.
src/Orleans.CodeGenerator/Model/MemberModel.cs Adds incremental serializable member model (field/property access strategy etc).
src/Orleans.CodeGenerator/Model/InvokableMethodProxyBase.cs Refactors to reference proxy generation context.
src/Orleans.CodeGenerator/Model/InvokableMethodDescription.cs Updates method-id/alias logic to use shared utilities/context.
src/Orleans.CodeGenerator/Model/GeneratedInvokableDescription.cs Updates compilation access via new context.
src/Orleans.CodeGenerator/MetadataGenerator.cs Major rewrite: metadata emission driven by aggregate models with deterministic ordering.
src/Orleans.CodeGenerator/InvokableGenerator.cs Refactors invokable generation to use proxy generation context + updated field descriptions.
src/Orleans.CodeGenerator/GeneratorServices.cs Introduces IGeneratorServices and implementation for shared generator dependencies.
src/Orleans.CodeGenerator/GeneratedCodeUtilities.cs Centralizes generated namespace, ids/aliases, hashed method-id, and codegen attributes.
src/Orleans.CodeGenerator/FieldIdAssignmentHelper.cs Enhances implicit/explicit field-id assignment validation and failure reporting.
src/Orleans.CodeGenerator/Diagnostics/CanNotGenerateImplicitFieldIdsDiagnostic.cs Fixes diagnostic message format placeholder usage.
src/Orleans.CodeGenerator/CopierGenerator.cs Refactors copier generation to use IGeneratorServices + shared utilities.
src/Orleans.CodeGenerator/CodeGeneratorOptions.cs Moves options into a dedicated type (separate from removed legacy generator).
src/Orleans.CodeGenerator/CodeGenerator.cs Removes legacy non-incremental generator implementation.
src/Orleans.CodeGenerator/ApplicationPartAttributeGenerator.cs Refactors attribute generation to be driven by provided inputs.
src/Orleans.CodeGenerator/ActivatorGenerator.cs Refactors activator generation to use IGeneratorServices + shared utilities.
Directory.Build.targets Prevents analyzer project references from being added as compile-time references.

Copilot's findings

Comments suppressed due to low confidence (1)

src/Orleans.CodeGenerator/FieldIdAssignmentHelper.cs:119

  • When a matching primary-constructor parameter is found, the code re-reads [Id] from the property (GetId(..., prop)) instead of from the constructor parameter (prm). That means an [Id] applied to the parameter will be ignored and the fallback ordinal will be used instead.
                else if (PropertyUtility.GetMatchingPrimaryConstructorParameter(prop, _constructorParameters) is { } prm)
                {
                    id = GeneratedCodeUtilities.GetId(_libraryTypes, prop);
                    if (id.HasValue)
                    {
                        _symbols[member] = (id.Value, true);
                    }
                    else
                    {
                        _symbols[member] = ((uint)_constructorParameters.IndexOf(prm), true);
                    }
  • Files reviewed: 51/53 changed files
  • Comments generated: 3

Comment thread test/Orleans.CodeGenerator.Tests/TestCompilationHelper.cs
Comment thread src/Orleans.CodeGenerator/ProxyGenerationContext.cs Outdated
Comment thread test/Orleans.CodeGenerator.Tests/DiagnosticTests.cs Outdated
ReubenBond and others added 7 commits April 28, 2026 15:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep explicit record positional member ids in the existing serialized member section so generated serializers remain compatible with previously written data. Continue treating implicit primary constructor members as constructor parameters and avoid duplicating field-targeted ids.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ReubenBond ReubenBond changed the title [WIP] Migrate to incremental source generator Migrate to incremental source generator Apr 30, 2026
@ReubenBond ReubenBond requested a review from Copilot April 30, 2026 00:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

@ReubenBond ReubenBond changed the title Migrate to incremental source generator perf(codegen): migrate to incremental source generator May 29, 2026
@Meir017

Meir017 commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

closes #9272

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants