Skip to content

Commit 240ee6c

Browse files
mthalmanmthalman
authored
Merge branch 'main' into darc-main-476ee6a5-de68-485b-811e-e4d207ff5a3e
2 parents 3c1d099 + 508d51a commit 240ee6c

399 files changed

Lines changed: 6788 additions & 2626 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.config/dotnet-tools.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,11 @@
33
"isRoot": true,
44
"tools": {
55
"microsoft.dnceng.secretmanager": {
6-
"version": "1.1.0-beta.25609.1",
6+
"version": "1.1.0-beta.26118.1",
77
"commands": [
88
"secret-manager"
99
],
1010
"rollForward": false
1111
}
1212
}
13-
}
13+
}

‎.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
*.json linguist-language=JSONC

‎.github/instructions/csharp.instructions.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
applyTo: "**/*.cs"
3+
---
4+
5+
# C#/.NET Guidelines
6+
7+
## Coding style
8+
9+
For all new C# code:
10+
11+
- Use file-scoped namespaces.
12+
- Use collection expresions - write `[1, 2, 3]` and not `new List<int> { 1, 2, 3 }`.
13+
- Use `var` for local variable declarations.
14+
- Use switch expressions and pattern matching.
15+
- Use string interpolation (`$"Hello, {name}!"`) instead of `string.Format` or concatenation.
16+
- Use `"""triple-quoted strings"""` for multi-line string literals. These can be interpolated as well.
17+
- Use expression-bodied members for simple getters and setters.
18+
19+
## Code Design Rules
20+
21+
- Use immutable records instead of classes for DTOs.
22+
- Do not default to `public` accessibility for members and classes. Follow the least-exposure rule: `private` > `internal` > `protected` > `public`
23+
- Do not add unused methods/parameters for use cases that were not asked for.
24+
- Reuse existing methods or services as much as possible.
25+
- Use composition over inheritance.
26+
- Use LINQ instead of for loops when working with collections.
27+
28+
## Error Handling & Edge Cases
29+
30+
- Guard early; use `string.IsNullOrWhiteSpace`, `ArgumentNullException.ThrowIfNull`, or `ArgumentNullException.ThrowIfNullOrWhiteSpace`.
31+
- Avoid using null for control flow.
32+
- In methods that return collections, return empty collections instead of null.
33+
- The null-forgiving operator (`!`) is **always** a code smell.
34+
- Do not add excessive or unnecessary try/catch blocks within the same assembly.
35+
36+
## Async Best Practices
37+
38+
- All async methods must have names ending with `Async`.
39+
- Always await async methods - do not "fire and forget".
40+
- Always accept and pass along a `CancellationToken` in async code.
41+
- Don’t add `async/await` if you can simply return a Task directly.
42+
43+
## Testing
44+
45+
- Use Shouldly when writing new assertions.
46+
- Use clear assertions that verify the outcome expressed by the test name.
47+
- Tests should be able to run in any order or in parallel.
48+
- Use or add helper methods for constructing mocks and complex test data objects.
49+
- Avoid disk I/O if possible; use in-memory alternatives or mocks.
50+
- Do not add "Arrange-Act-Assert" comments.
51+
52+
## Comments
53+
54+
- Add XML documentation comments for new **public** or **internal** types and members.
55+
- Primary documentation belongs on interfaces, implementations should use `<inheritdoc/>` unless additional context is needed.
56+
- Comments that simply restate the member or parameter name do not provide value.
57+
- Comments should provide additional context or explain non-obvious behavior, especially for parameters.
58+
- Comments inside methods should explain "why," not "what".
59+
- Avoid redundant comments that restate the code - not all code needs comments.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ msbuild.wrn
2626

2727
# .NET and tools directories
2828
.dotnet/
29+
.nuget/
2930
.packages/
3031
.tools/
3132

‎AGENTS.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# AGENTS.md
2+
3+
This file provides guidance to coding agents when working with code in this repository.
4+
5+
## Overview
6+
7+
This repository contains tooling for building and publishing Docker images in various .NET repos.
8+
The primary tool is **ImageBuilder**, a .NET CLI application that orchestrates Docker image builds based on manifest metadata files.
9+
10+
## Project Structure
11+
12+
This repo contains several projects:
13+
14+
- **`src/ImageBuilder/`** - The main ImageBuilder CLI tool (executable)
15+
- **`src/ImageBuilder.Models/`** - Shared model definitions for manifest files and image metadata
16+
- **`src/ImageBuilder.Tests/`** - Unit tests using xUnit, Moq, and Shouldly
17+
- **`eng/src/file-pusher/`** - Utility for pushing files to storage
18+
- **`eng/src/yaml-updater/`** - Utility for updating YAML files
19+
20+
## Building and Testing
21+
22+
- To build all projects, run `dotnet build`.
23+
- To run tests, run `dotnet test`.
24+
- If you run into permission issues during builds relating to copying `.dll` files, run `dotnet clean` and/or `build.sh -clean` and then build again.
25+
26+
### Build ImageBuilder Container Image
27+
28+
See [src/README.md](src/README.md) for instructions on building single-platform and multi-arch ImageBuilder container images locally.
29+
30+
## ImageBuilder Architecture
31+
32+
ImageBuilder is a command-based CLI tool that operates on manifest files.
33+
The manifest file (`manifest.json`) is the primary metadata source that defines which Docker images to build, their tags, platforms, and dependencies.
34+
35+
### Key Components
36+
37+
- **Commands** (`src/ImageBuilder/Commands/`) - Each command implements a specific operation (build, publish, generateBuildMatrix, etc.). Commands inherit from `Command<TOptions>` and use System.CommandLine for argument parsing.
38+
- **Models** (`src/ImageBuilder.Models/Manifest/`) - Defines the manifest schema and image metadata structures
39+
- **Services** - Abstracted services for Docker operations, Azure Container Registry interactions, Git operations, etc.
40+
41+
### Running ImageBuilder Locally
42+
43+
For quick validation during local development, use `dotnet run`:
44+
45+
```bash
46+
dotnet run --project src/ImageBuilder -- --help
47+
```
48+
49+
## `eng/docker-tools` Infrastructure
50+
51+
The `eng/docker-tools/` directory contains shared PowerShell scripts and Azure Pipelines templates used across all .NET Docker repositories.
52+
This repository is the source of truth for these files - changes made here are automatically synchronized to consuming repositories.
53+
54+
For comprehensive documentation on the docker-tools infrastructure, pipeline architecture, image building workflows, and troubleshooting, see [eng/docker-tools/DEV-GUIDE.md](eng/docker-tools/DEV-GUIDE.md).
55+
56+
## ImageBuilder Build and Deployment Workflow
57+
58+
ImageBuilder is published as a container image to the Microsoft Artifact Registry (MAR) at `mcr.microsoft.com/dotnet-buildtools/image-builder`.
59+
The ImageBuilder container image is defined in `src/manifest.json`.
60+
It is built from Dockerfiles `src/Dockerfile.linux` and `src/Dockerfile.windows`.
61+
62+
ImageBuilder uses itself to build its own container image.
63+
This means changes to ImageBuilder code and pipeline templates must be coordinated carefully in a two-step process.
64+
The version of ImageBuilder used by pipelines is specified in `eng/docker-tools/templates/variables/docker-images.yml` (`imageNames.imageBuilder` variable).
65+
66+
### Two-Step Change Process
67+
68+
When making changes that affect both ImageBuilder code and pipeline behavior:
69+
70+
1. **Step 1: Update ImageBuilder code**
71+
- Make code changes to ImageBuilder source
72+
- Test locally using `dotnet run --project src/ImageBuilder`
73+
- Propose changes in a pull request and wait for it to be merged and for CI to run.
74+
- A pull request will automatically be created that updates `docker-images.yml` with the new ImageBuilder tag.
75+
2. **Step 2: Update pipeline/manifest to use new ImageBuilder**
76+
- Once the new ImageBuilder image is published, pipeline templates in `eng/docker-tools/` can reference new features/commands
77+
- Consuming repositories will get the updated pipeline templates AND the new ImageBuilder image
78+
79+
### Why This Matters
80+
81+
- **Pipeline templates** in `eng/docker-tools/templates/` invoke ImageBuilder commands
82+
- These templates run using a specific version/tag of the ImageBuilder container image
83+
- If you add a new ImageBuilder command or change command behavior, pipelines can't use it until a new ImageBuilder image is published
84+
- This means code changes and pipeline template changes that depend on them must be done in separate steps
85+
86+
### Bootstrap Option for Development
87+
88+
To test ImageBuilder code changes and pipeline template changes together without waiting for the two-step process, use the `bootstrapImageBuilder` parameter in the unofficial pipeline (`eng/pipelines/dotnet-buildtools-image-builder-unofficial.yml`).
89+
90+
When `bootstrapImageBuilder: true`:
91+
92+
- ImageBuilder is built from source at the start of every pipeline job
93+
- The pipeline uses the freshly-built ImageBuilder instead of pulling from MCR
94+
- This allows validating ImageBuilder changes and pipeline template changes in a single pipeline run
95+
96+
Once changes are validated together via the bootstrap process, then changes can be proposed via the normal two-step change process described above.
97+
98+
## Key File Formats
99+
100+
### manifest.json (Input)
101+
102+
Manifest files define metadata about which Docker images ImageBuilder will build. The schema is defined by the `Manifest` model in `src/ImageBuilder.Models/Manifest/Manifest.cs`. For detailed documentation, see [documentation/manifest-file.md](documentation/manifest-file.md).
103+
104+
### image-info.json (Output)
105+
106+
Image info files are ImageBuilder's output that describe which images were built.
107+
The schema is defined by the `ImageArtifactDetails` model in `src/ImageBuilder.Models/Image/ImageArtifactDetails.cs`.
108+
These files contain:
109+
110+
- Image digests for each built platform
111+
- Tags applied to each image
112+
- Dockerfile paths and commit information
113+
- Build timestamps
114+
115+
Image info files are used by subsequent pipeline stages and are published to the versions repository to track what was built.
116+
See [eng/docker-tools/DEV-GUIDE.md](eng/docker-tools/DEV-GUIDE.md) for more details on how image info flows through pipelines.
117+
118+
## Documentation Maintenance
119+
120+
When making changes to ImageBuilder, pipeline templates, or infrastructure:
121+
122+
- Update [eng/docker-tools/DEV-GUIDE.md](eng/docker-tools/DEV-GUIDE.md) if you change pipeline architecture, workflows, or add new capabilities
123+
- Update [src/README.md](src/README.md) if you change how ImageBuilder container images are built
124+
- Update [documentation/manifest-file.md](documentation/manifest-file.md) if you modify the manifest schema
125+
- Update this file (AGENTS.md) if you add projects, change fundamental workflows, or modify architecture that affects how developers work with the codebase
126+
127+
Keep documentation synchronized with code changes so future developers have accurate guidance.

‎Directory.Build.props‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
<PropertyGroup Label="Build Settings">
66
<LangVersion>latest</LangVersion>
77
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
8+
<Nullable>enable</Nullable>
89
</PropertyGroup>
910

1011
<PropertyGroup Label="Packaging Settings">

‎documentation/images/request-signing-approval.png‎

58.8 KB
Loading

‎documentation/signing.md‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Container Image Signing
2+
3+
Container image signing uses ESRP via the MicroBuild signing plugin to sign Docker images with [Notary v2](https://notaryproject.dev/) signatures. When enabled, a Sign stage runs after images are built and before they are published.
4+
5+
## How To Enable Image Signing
6+
7+
### 1. Request Signing Approval
8+
9+
Request signing approval for your image publishing pipeline in the Azure DevOps GUI:
10+
11+
<img src="images/request-signing-approval.png" width="200">
12+
13+
Follow the instructions in the approval request form.
14+
15+
### 2. Enable Signing in Publish Configuration
16+
17+
Signing is controlled by the `Signing` property of ImageBuilder's `PublishConfiguration`. It can be
18+
configured in two different ways:
19+
20+
#### For .NET Repos
21+
22+
Pass the `enableSigning: true` parameter to `publish-config-prod.yml` or `publish-config-nonprod.yml`:
23+
24+
```yaml
25+
- template: /eng/docker-tools/templates/stages/dotnet/publish-config-prod.yml@self
26+
parameters:
27+
enableSigning: true
28+
# ... other parameters ...
29+
```
30+
31+
#### For Other Repos
32+
33+
If your pipeline constructs its own `publishConfig` object, add the `Signing` section to the configuration.
34+
The `Signing` property maps to the [`SigningConfiguration`](../src/ImageBuilder/Configuration/SigningConfiguration.cs) model.
35+
36+
```yaml
37+
publishConfig:
38+
# ... existing publish configuration ...
39+
Signing:
40+
Enabled: true
41+
# Get the appropriate ESRP signing keycode from the CSSC documentation:
42+
# https://aka.ms/cssc
43+
# Then look up the corresponding MicroBuild keycode in MicroBuild's source code:
44+
# https://devdiv.visualstudio.com/Engineering/_git/Sign?version=GBmain&path=/src/CertificateMappings.xml
45+
# These parameters refer to the MicroBuild signing keycode.
46+
ImageSigningKeyCode: 1234
47+
ReferrerSigningKeyCode: 5678
48+
# Important distinction: "test" signing is not supported on Linux (MicroBuild limitation).
49+
# For testing, use "real" signing with a testing certificate (from the CSSC documentation).
50+
SignType: real
51+
# Selects which set of root certificates to use when verifying signatures.
52+
# The Notation trust store configurations and certificates are baked into the ImageBuilder image.
53+
# Reference src/notation-trust/policies/ for available configurations.
54+
TrustStoreName: supplychain
55+
```
56+
57+
Additionally, the following variables must be available at pipeline run-time:
58+
59+
- `TeamName` - Team name registered with MicroBuild for signing.
60+
- `MicroBuildFeedSource` - NuGet feed source for the MicroBuild signing plugin.
61+
- `MicroBuildPluginVersion` - Use `latest` unless testing a pre-release version of the signing plugin.
62+
63+
See examples of these in [`variables/dotnet/common.yml`](../eng/docker-tools/templates/variables/dotnet/common.yml).

‎eng/Versions.props‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
<PropertyGroup>
66
<!-- Follow https://semver.org/spec/v2.0.0.html -->
77
<MajorVersion>0</MajorVersion>
8-
<MinorVersion>1</MinorVersion>
8+
<MinorVersion>2</MinorVersion>
99
<PatchVersion>0</PatchVersion>
1010
<VersionPrefix>$(MajorVersion).$(MinorVersion).$(PatchVersion)</VersionPrefix>
1111
<PreReleaseVersionLabel>beta</PreReleaseVersionLabel>

‎eng/docker-tools/CHANGELOG.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Docker Tools / ImageBuilder Changelog
2+
3+
All breaking changes and new features in `eng/docker-tools` will be documented in this file.
4+
5+
---
6+
7+
## 2026-03-04: Pre-build validation gated by `preBuildTestScriptPath` variable
8+
9+
The `PreBuildValidation` job condition now checks the new `preBuildTestScriptPath` variable instead of `testScriptPath`.
10+
This allows repos to independently control whether pre-build validation runs, without affecting functional tests.
11+
12+
The new variable defaults to `$(testScriptPath)`, so existing repos that have pre-build tests are not affected.
13+
Repos that do not have pre-build tests can set `preBuildTestScriptPath` to `""` to skip the job entirely.
14+
15+
---
16+
17+
## 2026-02-19: Separate Registry Endpoints from Authentication
18+
19+
- Pull request: [#1945](https://github.com/dotnet/docker-tools/pull/1945)
20+
- Issue: [#1914](https://github.com/dotnet/docker-tools/issues/1914)
21+
22+
Authentication details (`serviceConnection`, `resourceGroup`, `subscription`) have been moved from individual registry endpoints into a centralized `RegistryAuthentication` list.
23+
This fixes an issue where ACR authentication could fail when multiple service connections existed for the same registry.
24+
25+
**Before:** Each registry endpoint embedded its own authentication:
26+
27+
```yaml
28+
publishConfig:
29+
BuildRegistry:
30+
server: $(acr.server)
31+
repoPrefix: "my-prefix/"
32+
resourceGroup: $(resourceGroup)
33+
subscription: $(subscription)
34+
serviceConnection:
35+
name: $(serviceConnectionName)
36+
id: $(serviceConnection.id)
37+
clientId: $(serviceConnection.clientId)
38+
tenantId: $(tenant)
39+
PublishRegistry:
40+
server: $(acr.server)
41+
repoPrefix: "publish/"
42+
resourceGroup: $(resourceGroup)
43+
subscription: $(subscription)
44+
serviceConnection:
45+
name: $(publishServiceConnectionName)
46+
id: $(publishServiceConnection.id)
47+
clientId: $(publishServiceConnection.clientId)
48+
tenantId: $(tenant)
49+
```
50+
51+
**After:** Registry endpoints only contain `server` and `repoPrefix`. Authentication is centralized:
52+
53+
```yaml
54+
publishConfig:
55+
BuildRegistry:
56+
server: $(acr.server)
57+
repoPrefix: "my-prefix/"
58+
PublishRegistry:
59+
server: $(acr.server)
60+
repoPrefix: "publish/"
61+
RegistryAuthentication:
62+
- server: $(acr.server)
63+
resourceGroup: $(resourceGroup)
64+
subscription: $(subscription)
65+
serviceConnection:
66+
name: $(serviceConnectionName)
67+
id: $(serviceConnection.id)
68+
clientId: $(serviceConnection.clientId)
69+
tenantId: $(tenant)
70+
```
71+
72+
How to update:
73+
- Update any publishConfig parameters to match the new structure.
74+
- Multiple registries can share authentication. If two registries use the same ACR server, only one entry is needed in `RegistryAuthentication`.
75+
- The new structure should match [ImageBuilder's Configuration Model](https://github.com/dotnet/docker-tools/tree/a82572386854f15af441c50c6efa698a627e9f2b/src/ImageBuilder/Configuration).
76+
- Update service connection setup (if using `setup-service-connections.yml`):
77+
- The template now supports looking up service connections from `publishConfig.RegistryAuthentication`
78+
- Use the new `usesRegistries` parameter to specify which registries need auth setup:
79+
```yaml
80+
- template: eng/docker-tools/templates/stages/setup-service-connections.yml
81+
parameters:
82+
publishConfig: ${{ variables.publishConfig }}
83+
usesRegistries:
84+
- $(buildRegistry.server)
85+
- $(publishRegistry.server)
86+
```

0 commit comments

Comments
 (0)