Skip to content

Commit 594ee84

Browse files
chalew101CopilotWilliamBerryiii
authored
feat(security): enforce GitHub-hosted Ubuntu runner policy in CI (microsoft#2700)
# Pull Request ## Description Adds CI enforcement for the GitHub Actions `runs-on` runner policy, which was previously documented as a MUST rule in `workflows.instructions.md` but had no validator checking it. **This PR takes a different approach than the one proposed in the linked issue.** The issue was filed based on a PR review comment that flagged `ubuntu-slim` as a non-compliant runner label. During investigation, we confirmed `ubuntu-slim` is actually a GA (Jan 2026), GitHub-hosted, lightweight Ubuntu runner — it is compliant with the intended policy, not a violation. The real violation in that PR was `windows-latest`, which is genuinely disallowed. Rather than build a validator around the `ubuntu-slim` premise, this PR: * Clarifies `workflows.instructions.md` § Runners with an explicit **allow-list** (`ubuntu-latest`, dated `ubuntu-24.04`/`ubuntu-22.04` labels and their `-arm` variants, `ubuntu-slim`) and **disallow-list** (`windows-*`, `macos-*`, `self-hosted`, custom labels), so the rule is unambiguous about what is and isn't compliant. * Adds `scripts/security/Test-WorkflowRunner.ps1`, mirroring the existing `Test-WorkflowPermissions.ps1` validator pattern, to enforce that allow-list in CI against every job's `runs-on` value (including list-form and matrix-expression values, which are conservatively treated as violations since they can't be statically resolved). * Adds 28 Pester unit/integration tests for the new script. * Adds a reusable `workflow-runner-scan.yml` (mirroring `workflow-permissions-scan.yml`) and wires a `workflow-runner-check` job into `pr-validation.yml`'s required `pr-validation-success` gate. * Adds `npm run lint:workflow-runner` to `package.json` and the `validate:local` aggregate. This closes the broader gap the issue was really pointing at (an unenforced MUST rule) while fixing the underlying documentation ambiguity rather than special-casing `ubuntu-slim`. ## Related Issue(s) Closes microsoft#2696 ## Type of Change **Code & Documentation:** * [x] Documentation update **Infrastructure & Configuration:** * [x] GitHub Actions workflow * [x] Security configuration **AI Artifacts:** * [x] Reviewed contribution with `hve-builder` and addressed all actionable findings * [x] Copilot instructions (`.github/instructions/*.instructions.md`) **Other:** * [x] Script/automation (`.ps1`, `.sh`, `.py`) ## Sample Prompts (for AI Artifact Contributions) **User Request:** "Add a new job to this workflow with `runs-on: windows-latest`." **Execution Flow:** A Copilot session authoring/editing any `.github/workflows/*.yml` file has `workflows.instructions.md` in context (via its `applyTo` glob). It reads the clarified Runners allow/disallow list, flags `windows-latest` as non-compliant, and proposes an allow-listed replacement (e.g., `ubuntu-latest`). If asked what would catch this in CI, it cites `scripts/security/Test-WorkflowRunner.ps1`. **Output Artifacts:** No files are generated by the instructions themselves; they shape any workflow YAML a contributor authors. The `Test-WorkflowRunner.ps1` script produces JSON/SARIF/console reports, e.g.: ```json { "violations": [ { "file": ".github/workflows/example.yml", "job": "package-installer", "type": "NonUbuntuRunner", "value": "windows-latest" } ] } ``` **Success Indicators:** `npm run lint:workflow-runner` exits 0 with no violations; the `workflow-runner-check` job in `pr-validation.yml` passes and is included in the required `pr-validation-success` gate. ## Testing * `npm run test:ps -- -TestPath "scripts/tests/security/Test-WorkflowRunner.Tests.ps1"` — 28/28 passed. * Full `scripts/tests/security/` Pester suite — 643/643 passed. * Full repo Pester suite (`npm run test:ps`) — 4242 passed, 1 pre-existing/unrelated failure (environment-specific devcontainer-detection test on `main`, reproduced via `git stash`), 7 skipped. * `npm run lint:workflow-runner` against the real repo — 147/147 jobs pass. * `npm run lint:permissions`, `lint:dangerous-workflow`, `lint:pr-gate`, `lint:yaml`, `lint:json`, `lint:md`, `lint:frontmatter` — all pass. * Reviewed `.github/instructions/workflows.instructions.md` with `hve-builder` (review mode): static review found and fixed one High finding (Enforcement Statement was missing the new script); behavior gate executed via `hve-builder-tester` (simulation, Medium profile) — Pass, confirming a Copilot session correctly accepts `ubuntu-slim`, rejects/corrects `windows-latest`, and cites the enforcement script. ## Checklist ### Required Checks * [x] Documentation is updated (if applicable) * [x] Files follow existing naming conventions * [x] Changes are backwards compatible (if applicable) * [x] Tests added for new functionality (if applicable) ### AI Artifact Contributions * [x] Used `hve-builder` review mode to review contribution * [x] Addressed all actionable findings from the `hve-builder` review * [x] Verified contribution follows common standards and type-specific requirements ### Required Local Checks * [x] Local validation aggregate: `npm run validate:local` * [ ] Documentation validation (if docs changed): `npm run validate:docs` * [x] Spell checking: `npm run spell-check` * [x] Link validation: `npm run lint:md-links` ## Security Considerations * [x] This PR does not contain any sensitive or NDA information * [x] Any new dependencies have been reviewed for security issues * [x] Security-related scripts follow the principle of least privilege ## Additional Notes The issue as filed suggested the problem was `ubuntu-slim` being non-compliant. That premise is incorrect — `ubuntu-slim` is a valid GitHub-hosted Ubuntu runner. The actual gap (no CI enforcement of the runner policy at all) is real and is what this PR fixes, alongside clarifying the docs so this kind of misattribution doesn't recur. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Bill Berry <wberry@microsoft.com>
1 parent d1d6f21 commit 594ee84

7 files changed

Lines changed: 1206 additions & 3 deletions

File tree

.github/instructions/workflows.instructions.md

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,15 @@ Workflows MUST NOT persist GitHub credentials by default. Credential persistence
9898

9999
## Runners
100100

101-
Workflows MUST run on GitHub-hosted Ubuntu runners. Other runner types are not supported in hve-core.
101+
Workflows MUST run on GitHub-hosted Ubuntu runners. Windows, macOS, self-hosted, and other non-Ubuntu runner types are not supported in hve-core.
102+
103+
**Allowed `runs-on` labels** (GitHub-hosted Ubuntu images only):
104+
105+
* `ubuntu-latest`
106+
* `ubuntu-24.04`, `ubuntu-22.04` (and other GitHub-hosted Ubuntu version labels as they become available, including ARM variants such as `ubuntu-24.04-arm`)
107+
* `ubuntu-slim` (lightweight 1 vCPU GitHub-hosted runner; still Ubuntu, still GitHub-hosted)
108+
109+
**Disallowed `runs-on` values:** `windows-*`, `macos-*`, `self-hosted`, and any custom or third-party runner label.
102110

103111
**Required pattern:**
104112

@@ -268,6 +276,12 @@ All workflows MUST pass the following validation checks:
268276
* **What it enforces:** Every workflow declares a top-level `permissions:` block, and every job declares its own block unless the workflow-level block is empty
269277
* **CI blocking:** Failures block CI when configured to enforce compliance
270278

279+
### Runner Policy Validation
280+
281+
* **Script:** `scripts/security/Test-WorkflowRunner.ps1`
282+
* **What it enforces:** Every job's `runs-on` value is a GitHub-hosted Ubuntu label (see § Runners for the allow list)
283+
* **CI blocking:** Failures block CI when configured to enforce compliance
284+
271285
## Security Requirements
272286

273287
* Never expose secrets in logs or outputs
@@ -336,6 +350,7 @@ The following scripts enforce compliance:
336350
* `scripts/security/Test-DependencyPinning.ps1` - Validates dependency pinning
337351
* `scripts/security/Test-SHAStaleness.ps1` - Checks for stale dependencies
338352
* `scripts/security/Test-WorkflowPermissions.ps1` - Validates workflow permissions declarations
353+
* `scripts/security/Test-WorkflowRunner.ps1` - Validates `runs-on` values against the GitHub-hosted Ubuntu allow-list
339354
* `scripts/linting/Invoke-YamlLint.ps1` - Runs actionlint validation
340355
* `scripts/security/Test-PrValidationGate.ps1` - Validates the PR validation gate `needs:` completeness
341356

.github/workflows/pr-validation.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -420,6 +420,17 @@ jobs:
420420
upload-sarif: true
421421
upload-artifact: false
422422

423+
workflow-runner-check:
424+
name: Workflow Runner Check
425+
uses: ./.github/workflows/workflow-runner-scan.yml
426+
permissions:
427+
contents: read
428+
security-events: write # Required for SARIF upload to Security tab
429+
with:
430+
soft-fail: false
431+
upload-sarif: true
432+
upload-artifact: false
433+
423434
dangerous-workflow-check:
424435
name: Dangerous Workflow Check
425436
uses: ./.github/workflows/dangerous-workflow-scan.yml
@@ -686,6 +697,7 @@ jobs:
686697
- dependency-pinning-check
687698
- devcontainer-lockfile-check
688699
- workflow-permissions-check
700+
- workflow-runner-check
689701
- dangerous-workflow-check
690702
- action-version-consistency-scan
691703
- gitleaks-scan
Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Copyright (c) 2026 Microsoft Corporation. All rights reserved.
2+
# SPDX-License-Identifier: MIT
3+
4+
name: Workflow Runner Scan
5+
on:
6+
workflow_call:
7+
inputs:
8+
soft-fail:
9+
description: 'Continue on violations instead of failing'
10+
required: false
11+
type: boolean
12+
default: false
13+
upload-sarif:
14+
description: 'Upload SARIF results to GitHub Security tab'
15+
required: false
16+
type: boolean
17+
default: true
18+
upload-artifact:
19+
description: 'Upload results as workflow artifact'
20+
required: false
21+
type: boolean
22+
default: true
23+
outputs:
24+
compliance-score:
25+
description: 'Compliance score percentage'
26+
value: ${{ jobs.runner-check.outputs.compliance-score }}
27+
violation-count:
28+
description: 'Number of jobs not running on a GitHub-hosted Ubuntu runner'
29+
value: ${{ jobs.runner-check.outputs.violation-count }}
30+
is-compliant:
31+
description: 'Whether all jobs run on a GitHub-hosted Ubuntu runner'
32+
value: ${{ jobs.runner-check.outputs.is-compliant }}
33+
34+
permissions:
35+
contents: read
36+
37+
jobs:
38+
runner-check:
39+
name: Workflow Runner Check
40+
runs-on: ubuntu-latest
41+
permissions:
42+
contents: read
43+
security-events: write # Required for SARIF upload to Security tab
44+
outputs:
45+
compliance-score: ${{ steps.runner.outputs.compliance-score }}
46+
violation-count: ${{ steps.runner.outputs.violation-count }}
47+
is-compliant: ${{ steps.runner.outputs.is-compliant }}
48+
steps:
49+
- name: Checkout repository
50+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
51+
with:
52+
persist-credentials: false
53+
54+
- name: Setup PowerShell modules
55+
uses: ./.github/actions/setup-ps-modules
56+
57+
- name: Run Workflow Runner Validation
58+
id: runner
59+
shell: pwsh
60+
run: |
61+
New-Item -ItemType Directory -Force -Path logs | Out-Null
62+
$params = @{
63+
Path = '.github/workflows'
64+
Format = 'sarif'
65+
OutputPath = 'logs/workflow-runner-results.sarif'
66+
}
67+
if ('${{ inputs.soft-fail }}' -ne 'true') {
68+
$params['FailOnViolation'] = $true
69+
}
70+
./scripts/security/Test-WorkflowRunner.ps1 @params
71+
72+
# Write outputs for downstream consumers
73+
if (Test-Path 'logs/workflow-runner-results.sarif') {
74+
$sarif = Get-Content 'logs/workflow-runner-results.sarif' | ConvertFrom-Json
75+
$violationCount = @($sarif.runs[0].results).Count
76+
$isCompliant = $violationCount -eq 0
77+
78+
# Re-run with json format to extract compliance score
79+
$jsonParams = @{
80+
Path = '.github/workflows'
81+
Format = 'json'
82+
OutputPath = 'logs/workflow-runner-results.json'
83+
}
84+
./scripts/security/Test-WorkflowRunner.ps1 @jsonParams
85+
$report = Get-Content 'logs/workflow-runner-results.json' | ConvertFrom-Json
86+
$complianceScore = $report.ComplianceScore
87+
88+
"compliance-score=$complianceScore" >> $env:GITHUB_OUTPUT
89+
"violation-count=$violationCount" >> $env:GITHUB_OUTPUT
90+
"is-compliant=$($isCompliant.ToString().ToLower())" >> $env:GITHUB_OUTPUT
91+
}
92+
93+
- name: Upload SARIF results
94+
if: always() && inputs.upload-sarif
95+
uses: github/codeql-action/upload-sarif@ce729e4d353d580e6cacd6a8cf2921b72e5e310a # v3.27.0
96+
with:
97+
sarif_file: logs/workflow-runner-results.sarif
98+
category: workflow-runner
99+
continue-on-error: true
100+
101+
- name: Upload results artifact
102+
if: always() && inputs.upload-artifact
103+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
104+
with:
105+
name: workflow-runner-results
106+
path: logs/workflow-runner-results.*
107+
retention-days: 90
108+
continue-on-error: true

package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
"design-intent:schema:sync": "pwsh -NoProfile -File scripts/linting/Sync-DesignIntentSchema.ps1",
2626
"lint:version-consistency": "pwsh -NoProfile -File ./scripts/security/Test-ActionVersionConsistency.ps1 -FailOnMismatch -Format Json -OutputPath logs/action-version-consistency-results.json",
2727
"lint:permissions": "pwsh -NoProfile -File ./scripts/security/Test-WorkflowPermissions.ps1 -FailOnViolation",
28+
"lint:workflow-runner": "pwsh -NoProfile -File ./scripts/security/Test-WorkflowRunner.ps1 -FailOnViolation",
2829
"lint:extension-artifact-naming": "pwsh -NoProfile -File scripts/linting/Test-ExtensionArtifactNaming.ps1",
2930
"lint:dangerous-workflow": "pwsh -NoProfile -File ./scripts/security/Test-DangerousWorkflow.ps1 -FailOnViolation",
3031
"lint:dependency-pinning": "pwsh -NoProfile -File ./scripts/security/Test-DependencyPinning.ps1 -FailOnUnpinned",
@@ -42,7 +43,7 @@
4243
"lint:asset-docs": "pwsh -NoProfile -File scripts/linting/Validate-AssetDocs.ps1 -FailOnMissing -CheckSync",
4344
"lint:models": "pwsh -NoProfile -File scripts/linting/Test-ModelReferences.ps1 -OutputPath logs/model-validation-results.json",
4445
"lint:models:refresh": "pwsh -NoProfile -File scripts/linting/Update-ModelCatalog.ps1",
45-
"validate:local": "npm run lint:plugin-manifest && npm run lint:tables && npm run lint:md && npm run lint:ps && npm run lint:yaml && npm run lint:json && npm run lint:links && npm run lint:md-links && npm run lint:frontmatter && npm run lint:adr-consistency && npm run lint:hooks && npm run lint:design-intent && npm run lint:version-consistency && npm run lint:permissions && npm run lint:dangerous-workflow && npm run lint:dependency-pinning && npm run lint:public-dependency-feeds && npm run lint:pr-gate && npm run lint:ps-module-pins && npm run lint:extension-artifact-naming && npm run lint:py && npm run validate:skills && npm run lint:ai-artifacts && npm run lint:asset-docs && npm run lint:models && npm run validate:devcontainer-lockfile",
46+
"validate:local": "npm run lint:plugin-manifest && npm run lint:tables && npm run lint:md && npm run lint:ps && npm run lint:yaml && npm run lint:json && npm run lint:links && npm run lint:md-links && npm run lint:frontmatter && npm run lint:adr-consistency && npm run lint:hooks && npm run lint:design-intent && npm run lint:version-consistency && npm run lint:permissions && npm run lint:workflow-runner && npm run lint:dangerous-workflow && npm run lint:dependency-pinning && npm run lint:public-dependency-feeds && npm run lint:pr-gate && npm run lint:ps-module-pins && npm run lint:extension-artifact-naming && npm run lint:py && npm run validate:skills && npm run lint:ai-artifacts && npm run lint:asset-docs && npm run lint:models && npm run validate:devcontainer-lockfile",
4647
"lint:tables": "pwsh -NoProfile -File scripts/linting/Format-MarkdownTables.ps1 -Check",
4748
"format:tables": "pwsh -NoProfile -File scripts/linting/Format-MarkdownTables.ps1",
4849
"extension:prepare": "pwsh ./scripts/extension/Prepare-Extension.ps1 && npm run extension:postprocess",

scripts/security/Modules/SecurityClasses.psm1

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ class DependencyViolation {
3636
- MissingVersionComment: Dependency is pinned but lacks a human-readable version comment
3737
- MissingPermissions: Workflow file lacks required permissions declarations
3838
- MissingJobPermissions: Workflow job lacks its own permissions declaration and inherits an implicit grant
39+
- NonUbuntuRunner: Job's runs-on value is not a GitHub-hosted Ubuntu label
40+
- MissingRunner: Job has no resolvable runs-on value
3941
- Empty string: Default or unclassified violation
4042
#>
4143

@@ -47,7 +49,7 @@ class DependencyViolation {
4749
[string]$CurrentRef
4850
[ValidateSet('High', 'Medium', 'Low', 'Info')]
4951
[string]$Severity
50-
[ValidateSet('Unpinned', 'Stale', 'VersionMismatch', 'MissingVersionComment', 'MissingPermissions', 'MissingJobPermissions', '')]
52+
[ValidateSet('Unpinned', 'Stale', 'VersionMismatch', 'MissingVersionComment', 'MissingPermissions', 'MissingJobPermissions', 'NonUbuntuRunner', 'MissingRunner', '')]
5153
[string]$ViolationType
5254
[string]$Description
5355
[string]$Remediation

0 commit comments

Comments
 (0)