Merge branch 'main' into rayz/v3-datadog-only-default

Author: rayz

.claude/skills/run-windows-e2e/SKILL.md

@@ -0,0 +1,111 @@
+---
+name: run-windows-e2e
+description: Run Windows E2E tests (MSI install tests or Fleet Automation/installer tests) locally against AWS-provisioned VMs
+allowed-tools: Bash, Read, Glob, Grep, AskUserQuestion
+argument-hint: "[suite] [TestFunctionName] [--build release|pipeline|local] [--pipeline-id <id>] [--version <version>]"
+---
+
+Run Windows E2E tests from `test/new-e2e/tests/windows/` or `test/new-e2e/tests/installer/windows/`.
+
+Detailed reference material lives in `references/` next to this file — read the
+relevant one when a step calls for it rather than duplicating it here:
+
+- [`references/setup.md`](references/setup.md) — prerequisites, `~/.test_infra_config.yaml`, dev mode
+- [`references/running.md`](references/running.md) — `setup-env`, local builds, `go test` flags
+- [`references/vm-access.md`](references/vm-access.md) — connecting to a dev-mode VM (RDP/SSH)
+- [`references/troubleshooting.md`](references/troubleshooting.md) — test outputs, Pulumi locks, AWS profile
+
+## Instructions
+
+### Step 1 — Parse `$ARGUMENTS`
+
+Determine:
+- **Suite**: which test suite to run (e.g. `install-test`, `service-test`, `agent-package`, `install-script`). If not provided, ask the user.
+- **Test function**: specific `TestXxx` function. Most suites expect exactly one test per run — ask the user which one if not specified.
+- **Artifact source**: `--build pipeline` (default), `--build local`, or `--build release`; pass `--pipeline-id <id>` through if given.
+- **Stable/previous version** (upgrade tests only): if the user specifies a version to upgrade *from*, plan a second `setup-env` run with `--prefix STABLE_AGENT` in Step 3 (see [`references/running.md`](references/running.md) "Upgrade tests").
+- **Branch**: if the user mentions a branch ("from main"), pass `--branch <name>` to `setup-env`. The default is the current git branch, which may have no pipelines if it's a local feature branch.
+
+Map suite names to Go package paths:
+
+| Suite | Package path |
+|-------|-------------|
+| `install-test` | `./test/new-e2e/tests/windows/install-test` |
+| `service-test` | `./test/new-e2e/tests/windows/service-test` |
+| `fips-test` | `./test/new-e2e/tests/windows/fips-test` |
+| `domain-test` | `./test/new-e2e/tests/windows/domain-test` |
+| installer / Fleet Automation (agent-package, install-script, install-exe, ddot, apm-inject, …) | `./test/new-e2e/tests/installer/windows` |
+
+The installer / Fleet Automation tests are all one flat package (the
+`suites/<package>/` subdirectories were flattened in #47161) — pick the area by
+test function with `-run` (e.g. `TestAgentUpgrades`, `TestInstallScript`,
+`TestDDOTExtensionViaMSI`, `TestAPMInjectInstalls`).
+
+If the user gives a partial name or test function, search with Glob/Grep under `test/new-e2e/tests/windows/` and `test/new-e2e/tests/installer/windows/` to resolve it.
+
+### Step 2 — Check prerequisites
+
+```bash
+test -f ~/.test_infra_config.yaml && echo "EXISTS" || echo "MISSING"
+pulumi version 2>/dev/null || echo "MISSING"
+```
+
+If either is missing, offer to run `dda inv e2e.setup` and wait for the user to
+complete its interactive prompts. If prerequisites exist but `devMode` is not
+set, mention that `devMode: true` reuses VMs across runs (much faster for
+iterative development). Full detail in [`references/setup.md`](references/setup.md).
+
+### Step 3 — Resolve artifact environment variables
+
+Run `setup-env` with `--fmt json` to capture the required env vars (no shell
+`eval` needed — prepend the parsed pairs inline to `go test` in Step 5).
+
+```bash
+# From a pipeline (most common)
+dda inv new-e2e-tests.setup-env --build pipeline --fmt json [--branch <branch>] [--pipeline-id <id>]
+
+# From a local build (run `dda inv msi.build` first, + `msi.package-oci` for installer/OCI tests)
+dda inv new-e2e-tests.setup-env --build local --fmt json
+```
+
+For upgrade tests, run a second time with `--prefix STABLE_AGENT` and merge the
+vars in. GitLab token handling, local-build details, and the `STABLE_AGENT`
+flow are in [`references/running.md`](references/running.md).
+
+### Step 4 — Check for stale state (dev mode only)
+
+If `devMode: true` and the user is rerunning, the previous VM may still have the
+agent installed. Ask whether they've cleaned up (MSI tests: uninstall the agent;
+installer tests: `datadog-installer.exe purge`). See
+[`references/running.md`](references/running.md) "Clean state between runs".
+
+### Step 5 — Build and confirm the `go test` command
+
+```bash
+go test -v -timeout 30m -tags test <package-path> -run <TestFunction>$
+```
+
+Two rules to apply (rationale in [`references/running.md`](references/running.md)):
+anchor the `-run` regex with `$` at both suite and subtest level, and use the
+exact package path with **no trailing `/...`** (or output won't stream).
+
+Show the full command to the user and confirm before running.
+
+### Step 6 — Run the test
+
+Warn the user that AWS SSO auth may open a browser window when the test starts
+(the test pauses until login completes), and that a non-sandbox `AWS_PROFILE`
+will cause auth errors — advise `unset AWS_PROFILE`.
+
+Run with `run_in_background: true` since tests provision real AWS VMs.
+Provisioning takes a few minutes; once the VM is up, SSH becomes available
+within ~60s (Linux) / ~180s (Windows). If SSH is not available within those
+windows, troubleshoot before assuming the test is still running normally
+(see [`references/vm-access.md`](references/vm-access.md)).
+
+### Step 7 — Report results
+
+When the test completes:
+- Report pass/fail.
+- On failure, point the user to `~/e2e-output/latest/` (crash dumps, agent/installer logs, event logs). If `devMode` is on, the VM is still up — offer to help RDP/SSH in via [`references/vm-access.md`](references/vm-access.md).
+- For Pulumi lock errors or AWS auth errors, see [`references/troubleshooting.md`](references/troubleshooting.md).

.claude/skills/run-windows-e2e/references/running.md

@@ -0,0 +1,101 @@
+# Artifacts & running — Windows E2E Tests
+
+How to resolve which MSI/OCI package the tests use, and how to invoke
+`go test`. See [setup.md](setup.md) for one-time environment setup.
+
+## Setting up artifact environment variables
+
+Windows tests need to know which MSI and/or OCI package to use. The `setup-env`
+helper configures the required environment variables. Use `--fmt json` when
+driving it programmatically (parse the `"VAR": "value"` map and prepend the
+pairs inline to `go test`); use `--fmt powershell | Invoke-Expression` or
+`eval "$(...)"` for an interactive shell.
+
+### From a pipeline (most common)
+
+Auto-detects the most recent successful pipeline on the current branch. Add
+`--branch main` to target `main` instead — useful when the current branch is a
+local feature branch with no pipelines. Add `--pipeline-id <id>` to pin a
+specific pipeline.
+
+```bash
+# Programmatic (skill default): parse the "VAR": "value" map, prepend inline to `go test`
+dda inv new-e2e-tests.setup-env --build pipeline --fmt json [--branch <branch>] [--pipeline-id <id>]
+
+# Interactive shell — PowerShell
+dda inv new-e2e-tests.setup-env --build pipeline --fmt powershell | Invoke-Expression
+
+# Interactive shell — Bash/WSL
+eval "$(dda inv new-e2e-tests.setup-env --build pipeline)"
+```
+
+The task auto-resolves a GitLab token via `ddtool` if configured. If that fails
+it asks for `GITLAB_TOKEN`; retry with `GITLAB_TOKEN=<token> dda inv new-e2e-tests.setup-env ...`.
+
+### From a local build
+
+Build the MSI first (add `msi.package-oci` only for installer/OCI tests):
+
+```bash
+dda inv msi.build          # -> omnibus/pkg/datadog-agent-<version>-x86_64.msi
+dda inv msi.package-oci     # -> omnibus/pkg/datadog-agent-<version>-windows-amd64.oci.tar
+dda inv new-e2e-tests.setup-env --build local --fmt json
+```
+
+### Upgrade tests (stable / previous version)
+
+Upgrade tests need a second package resolved under the `STABLE_AGENT_` prefix.
+Run `setup-env` again with `--prefix STABLE_AGENT --build <mode>` (same modes:
+`pipeline`, `local`, `release`) plus any relevant `--version` / `--pipeline-id`
+flags, and merge those vars in, overriding the first run's defaults.
+
+The variables themselves are documented per suite — see
+`test/new-e2e/tests/windows/AGENTS.md` ("Package resolution") and
+`test/new-e2e/tests/installer/windows/AGENTS.md` ("Package resolution").
+
+## Running a specific suite
+
+```bash
+go test -v -timeout 30m -tags test <package-path> -run <TestFunction>$
+```
+
+Two rules that are easy to get wrong:
+
+- **Anchor the `-run` regex with `$`**, at both the suite and subtest level. The
+  flag is a Go regex; without the anchor `TestInstall` also matches
+  `TestInstallOpts`/`TestInstallFail`, and `TestUpgradeAgentPackage` also
+  matches `TestUpgradeAgentPackageOCIBootstrap`.
+- **Use the exact package path, no trailing `/...`.** With `/...` Go buffers all
+  output per-package and prints it only when the package finishes, making the
+  test look silent. Without it, output streams live.
+
+Examples:
+
+```bash
+# MSI install test
+go test -v -timeout 30m -tags test ./test/new-e2e/tests/windows/install-test -run TestInstall$
+
+# Service lifecycle test
+go test -v -timeout 30m -tags test ./test/new-e2e/tests/windows/service-test -run TestServiceBehaviorPowerShell$
+
+# Fleet Automation — agent package, specific subtest
+go test -v -timeout 30m -tags test ./test/new-e2e/tests/installer/windows -run "TestAgentUpgrades$/TestUpgradeAgentPackage$"
+```
+
+See each suite's `AGENTS.md` for the full list of test functions and the CI job
+names that map to them.
+
+AWS SSO authentication may open a browser window when the test starts; the test
+pauses until login completes. This is expected. If `AWS_PROFILE` is set to a
+non-sandbox profile it causes auth errors — see [troubleshooting.md](troubleshooting.md).
+
+## Clean state between runs
+
+Most tests install the agent as part of the test and expect a clean starting
+state, so run them one at a time. In dev mode (reused VM), clean up between runs
+so state does not leak:
+
+- **MSI install tests** (`install-test/`, `domain-test/`, `fips-test/`):
+  uninstall via Add/Remove Programs or `MsiExec.exe /x {product-code}` on the VM.
+- **Installer / Fleet Automation tests** (`installer/windows/`): run
+  `datadog-installer.exe purge` on the VM.

.claude/skills/run-windows-e2e/references/setup.md

@@ -0,0 +1,56 @@
+# Setup — Windows E2E Tests
+
+One-time environment setup for running Windows E2E tests locally. Applies to
+both MSI install tests (`test/new-e2e/tests/windows/`) and Fleet Automation /
+installer tests (`test/new-e2e/tests/installer/windows/`).
+
+## Prerequisites
+
+Follow the general E2E setup guide first:
+[docs/public/how-to/test/e2e.md](../../../../docs/public/how-to/test/e2e.md).
+It covers AWS `agent-sandbox` account access, `aws-vault` setup, the
+`PULUMI_CONFIG_PASSPHRASE` requirement, and required tooling (Go, Python,
+`dda`).
+
+The Windows tests provision real AWS EC2 Windows instances and require the same
+AWS access as all other E2E tests. Without valid AWS credentials and
+`PULUMI_CONFIG_PASSPHRASE` set, the run fails before any test code executes.
+
+`dda inv e2e.setup` (defined in `tasks/e2e_framework/setup/`) walks through this
+interactively: it validates CLI tools, installs Pulumi if missing, prompts for
+AWS keypair paths, Datadog API key, and Pulumi passphrase, then writes
+`~/.test_infra_config.yaml`.
+
+## Persistent configuration
+
+Most run parameters are easier to set once in `~/.test_infra_config.yaml` than
+as environment variables each session. Relevant fields:
+
+- AWS keypair paths (`privateKeyPath` is also the SSH key — see
+  [vm-access.md](vm-access.md))
+- Datadog API key — **required**; the agent MSI install needs a valid key
+- Pulumi passphrase — alternative to the `PULUMI_CONFIG_PASSPHRASE` env var
+- `devMode` — see below
+
+## Dev mode
+
+By default each run provisions a fresh VM and destroys it afterwards, which
+makes local test-fix-rerun loops slow. Set `devMode: true` in
+`~/.test_infra_config.yaml` to reuse the same VM across runs. The VM is
+identified by the Pulumi stack name, derived from the test package and suite
+name.
+
+Per-run alternative:
+
+```bash
+E2E_DEV_MODE=true go test -v -timeout 30m -tags test ./test/new-e2e/tests/windows/install-test -run TestInstall$
+```
+
+Or in VSCode:
+
+```json
+"go.testEnvVars": { "E2E_DEV_MODE": "true" }
+```
+
+When reusing a VM, clean up the agent between runs so state does not leak — see
+[running.md](running.md) ("Clean state between runs").

.claude/skills/run-windows-e2e/references/troubleshooting.md

@@ -0,0 +1,52 @@
+# Troubleshooting — Windows E2E Tests
+
+## Test outputs
+
+Test output files (logs, crash dumps, event logs, agent logs) are written via
+`runner.GetProfile().CreateOutputSubDir()` to:
+
+```
+~/e2e-output/<suite-name>/<timestamp>/
+~/e2e-output/latest/                    ← symlink to the most recent run
+```
+
+When a test fails, the suite's `AfterTest` hook automatically collects
+diagnostics into the output directory — WER crash dumps, Windows event logs
+(`.evtx`), agent logs, and installer logs depending on the suite. Check
+`~/e2e-output/latest/` first when investigating a failure. If `devMode` is on,
+the VM is still up — connect to it via [vm-access.md](vm-access.md).
+
+In CI, outputs are stored as GitLab job artifacts, accessible from the job's
+artifact browser in the pipeline UI.
+
+## Pulumi lock errors
+
+If you see `error: the stack is currently locked by 1 lock(s)`:
+
+```bash
+dda inv new-e2e-tests.clean            # remove local Pulumi locks
+dda inv new-e2e-tests.clean -s         # also clean local stack state
+dda inv new-e2e-tests.clean --output   # clear local test output
+```
+
+If cleanup reports "Cleanup supported for local state only":
+
+```bash
+pulumi login --local
+```
+
+## AWS account / profile
+
+Local runs must use the `agent-sandbox` AWS account (CI uses `agent-qa`). The
+e2e framework automatically uses the `sso-agent-sandbox-account-admin` profile,
+so no explicit `aws-vault` invocation is needed.
+
+However, if `AWS_PROFILE` is already set to a different profile it overrides the
+automatic selection and causes auth errors. Unset it before running:
+
+```bash
+unset AWS_PROFILE
+```
+
+A common symptom of the wrong account:
+`User: arn:aws:sts::... is not authorized to perform: ecr:BatchGetImage`.

.claude/skills/run-windows-e2e/references/vm-access.md

@@ -0,0 +1,59 @@
+# Accessing the test VM — Windows E2E Tests
+
+When `devMode` is enabled (see [setup.md](setup.md)) the VM persists after the
+test, so you can RDP or SSH in to investigate.
+
+## Find the stack name
+
+Two ways:
+
+- From test output: look for `Creating workspace for stack: user-e2e-<suitename>-<hash>`
+- From the CLI: `pulumi stack ls --all` — find the entry matching the suite name
+
+Format: `user-e2e-<suitename-lowercase>-<hash>`.
+
+## Get connection details
+
+```bash
+pulumi stack --stack organization/e2elocal/<stack-name> output --json --show-secrets
+```
+
+The output includes `address` (private IP), `username` (`Administrator`),
+`password`, and an `osFamily` integer. The OS family/flavor enum values are in
+`test/e2e-framework/components/os/const.go` (`WindowsFamily = 2`,
+`WindowsServer = 509`, `WindowsClient = 510`).
+
+## RDP
+
+```bash
+# Opens RDP, prints IP + password, copies password to clipboard
+dda inv aws.rdp-vm --stack-name <stack-name>
+```
+
+`dda inv aws.show-vm` does **not** work for e2e test stacks — it is only for VMs
+created with `dda inv aws.create-vm` (it prepends the username twice and looks
+for the wrong output key, `"aws-vm"` instead of `"dd-Host-aws-vm"`).
+
+## SSH
+
+The SSH key is the `privateKeyPath` from `~/.test_infra_config.yaml`.
+
+```bash
+ssh -i <privateKeyPath> -o StrictHostKeyChecking=no <username>@<ip> "<commands>"
+```
+
+Pick the command separator from `osFamily` in the stack output:
+
+- `osFamily: 1` = Linux → `&&` or `;` (bash)
+- `osFamily: 2` = Windows → `;` only (PowerShell over SSH; `&&` is invalid)
+
+### SSH availability
+
+SSH should be reachable within ~60s for Linux VMs and ~180s for Windows VMs
+after the VM is provisioned. If it takes longer:
+
+- Make sure the key is loaded in your SSH agent (`ssh-add -l`) or pass `-i`
+  explicitly.
+- Verify the keypair name in `~/.test_infra_config.yaml` matches the AWS key
+  pair used to provision the VM — a mismatch means the wrong public key was
+  injected and authentication will always fail.

.github/CODEOWNERS

@@ -39,6 +39,7 @@
 /.claude/skills/review-pr-comments/              @DataDog/agent-devx
 /.claude/skills/run-e2e/                         @DataDog/agent-devx
 /.claude/skills/run-jira/                        @DataDog/agent-devx
+/.claude/skills/run-windows-e2e/                 @DataDog/windows-products
 /.claude/skills/update-3rd-party-libs            @DataDog/agent-build
 /.claude/skills/update-otel-deps/                @DataDog/opentelemetry-agent
 /.claude/skills/write-e2e/                       @DataDog/agent-devx
@@ -456,6 +457,7 @@
 # Per-team ownership of individual event platform pipeline descriptor files.
 # Each file holds the passthroughPipelineDesc entries for one owning team so
 # that teams can merge pipeline config changes without Logs team review.
+/comp/forwarder/eventplatform/impl/pipelines_agentdiscovery.go         @DataDog/agent-discovery
 /comp/forwarder/eventplatform/impl/pipelines_dbm.go                    @DataDog/database-monitoring
 /comp/forwarder/eventplatform/impl/pipelines_ndm_core.go               @DataDog/network-device-monitoring-core
 /comp/forwarder/eventplatform/impl/pipelines_ndm_integrations.go       @DataDog/ndm-integrations