Skip to content

Commit 84381ef

Browse files
PerishCodeLooper
andauthored
feat(ci): add global hash skip planning (#7185)
* feat(ci): add global hash skip planning * refactor(ci): remove superseded hash skip cache * refactor(ci): keep scope trust in planner * fix(ci): run scope goldens for planner changes Generated-By: looper 0.11.2 (runner=fixer, agent=codex) * fix(ci): publish hash state after successful validation Generated-By: looper 0.11.2 (runner=fixer, agent=codex) --------- Co-authored-by: Looper <looper@noreply.github.com>
1 parent 4fccbdb commit 84381ef

28 files changed

Lines changed: 1587 additions & 3721 deletions

.github/AGENTS.md

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,9 @@ Before changing GitHub automation, read the current versions of:
1111
- `.github/workflows/autofix.atom.yml`
1212
- `.github/workflows/report.atom.yml`
1313
- `.github/scripts/handoff.py`
14-
- `scripts/scopes.ts`
15-
- `specs/current/ci.md` when changing scope rules, confidence tiers, or guards
14+
- `.github/config/runners.json`, `.github/config/scopes.json`, and `.github/config/hash.json`
15+
- `.github/scripts/runners.py`, `.github/scripts/scopes.py`, and `.github/scripts/hash.py`
16+
- `specs/current/ci.md` when changing scope rules, confidence tiers, or planner invariants
1617
- `e2e/tests/packaged-smoke-workflow.test.ts`
1718
- `scripts/approve-fork-pr-workflows.ts` and `e2e/tests/scripts/approve-fork-pr-workflows.test.ts` when touching fork PR approval behavior
1819

@@ -26,7 +27,7 @@ Business layer:
2627

2728
- Business workflows decide what happened and what should be requested next.
2829
- `ci.yml` is the main low-privilege PR, merge-queue, and manual validation gate (application merge bar only).
29-
- `ci.yml` should run validation, decide scopes, and produce typed handoff artifacts.
30+
- `ci.yml` should resolve runners, compose scope and hash decisions in its Linux `plan` job, run validation, and produce typed handoff artifacts.
3031
- Packaging checks are standalone and outside the merge gate: `nix.yml` (flake check) and `docker-image.yml` (image validate + publish). Do not re-attach them to `Validate workspace`.
3132
- Business workflows should not perform trusted writes to PR comments or branches when a capability workflow can do it.
3233

@@ -51,6 +52,21 @@ Default rule: do not add a new domain-specific follow-on workflow such as `foo.c
5152

5253
New workflow-owned helpers should usually live under `.github/scripts/`. Prefer TypeScript for project-owned scripts in general, but Python is acceptable for small GitHub runner glue when stdlib portability and low setup cost matter. Keep such exceptions narrow and covered by `pnpm guard` policy.
5354

55+
The CI control plane is deliberately Linux-only and stdlib-only. Runner classes,
56+
scope rules, and hash declarations live in `.github/config/`; their Python
57+
entrypoints initialize metadata before workload runners start. A Windows job
58+
must never invoke these scripts. Keep the four layers independent: runner
59+
placement, changed-file scopes, input hashes, and fine-grained commands inside
60+
a workload.
61+
62+
`hash.py` is a static comparison register, not a success cache. It reads the
63+
previous identity-to-hash map restored by Actions cache, computes the current
64+
map from Git inputs, and replaces the local state immediately. The plan carries
65+
that pending map to `validate`, which publishes it only after the gate succeeds;
66+
a failed run therefore cannot authorize identical-input skips on a fresh retry.
67+
Only a workload's YAML `if` gives the comparison skip semantics; cache loss or
68+
corruption starts cold.
69+
5470
## Handoff contract
5571

5672
Use `.github/scripts/handoff.py` for all CI follow-on artifact names and paths. The canonical layout is:
@@ -125,13 +141,14 @@ Keep `.github/workflows/ci.yml` as the only approved workflow path unless a main
125141
- Same-repo patch: produce `handoff/autofix` and let `autofix.atom.yml` consume it.
126142
- Rich/generated comment: produce `handoff/report` and let `report.atom.yml` materialize and upsert it.
127143
- New naming, paths, or metadata: update `.github/scripts/handoff.py`.
128-
2. Update scope routing in `scripts/scopes.ts` when a workflow/script should trigger a validation lane.
129-
3. Update topology coverage in `e2e/tests/packaged-smoke-workflow.test.ts` or the relevant script test.
130-
4. Run the focused checks:
144+
2. Update scope routing in `.github/config/scopes.json`, then run `python3 .github/scripts/scopes.py validate`.
145+
3. Declare workload input closure in `.github/config/hash.json`; use `"*"` until a narrower set has high-confidence evidence.
146+
4. Update topology coverage in `e2e/tests/packaged-smoke-workflow.test.ts` or the relevant script test.
147+
5. Run the focused checks:
131148
- `python3 .github/scripts/handoff.py self-check`
132149
- `actionlint -color`
133150
- `pnpm --filter @open-design/e2e test tests/packaged-smoke-workflow.test.ts`
134-
5. Run repo-level checks before handing off:
151+
6. Run repo-level checks before handing off:
135152
- `pnpm guard`
136153
- `pnpm typecheck`
137154

.github/actions/hash-skip/action.yml

Lines changed: 0 additions & 44 deletions
This file was deleted.

.github/config/hash.json

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
{
2+
"schema": { "version": 1 },
3+
"suites": {
4+
"ci-control": [
5+
".github/config/hash.json",
6+
".github/config/scopes.json",
7+
".github/scripts/hash.py",
8+
".github/scripts/scopes.py",
9+
".github/scripts/lib/",
10+
".github/workflows/ci.yml"
11+
],
12+
"web": [
13+
"apps/web/", "packages/components/", "packages/contracts/", "packages/host/",
14+
"packages/platform/", "packages/release/", "packages/sidecar/", "packages/sidecar-proto/",
15+
"package.json", "pnpm-lock.yaml", "pnpm-workspace.yaml"
16+
],
17+
"tools-pack": [
18+
"tools/pack/", "apps/packaged/", "apps/desktop/", "packages/components/", "packages/host/",
19+
"packages/platform/", "packages/release/", "packages/sidecar/", "packages/sidecar-proto/",
20+
"package.json", "pnpm-lock.yaml", "pnpm-workspace.yaml"
21+
],
22+
"ui": [
23+
"suite://web", "apps/daemon/", "e2e/", ".github/actions/setup-playwright/",
24+
".github/actions/setup-workspace/"
25+
]
26+
},
27+
"workflows": {
28+
"ci": {
29+
"static_gate": ["*"],
30+
"preflight": ["*"],
31+
"workspace_unit_tests": ["*"],
32+
"daemon_unit_tests": ["*"],
33+
"windows_tools_pack_payload_tests": ["*"],
34+
"web_workspace_tests": ["*"],
35+
"e2e_vitest": ["*"],
36+
"playwright_critical": ["*"],
37+
"ui_p0": ["*"],
38+
"playwright_visual": ["*"]
39+
}
40+
}
41+
}

.github/config/runners.json

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
{
2+
"schema": { "version": 1 },
3+
"defaultMode": "default",
4+
"aliases": { "performance": "default" },
5+
"profiles": {
6+
"github.ubuntu": ["ubuntu-24.04"],
7+
"github.windows": ["windows-latest"],
8+
"blacksmith.small": ["blacksmith-4vcpu-ubuntu-2404"],
9+
"blacksmith.large": ["blacksmith-8vcpu-ubuntu-2404"],
10+
"nexu.small": ["nexu-runners-small"],
11+
"nexu.medium": ["nexu-runners-medium"],
12+
"nexu.large": ["nexu-runners-large"],
13+
"nexu.xlarge": ["nexu-runners-xlarge"]
14+
},
15+
"modes": {
16+
"default": {
17+
"control": "nexu.small",
18+
"general_medium": "nexu.medium",
19+
"workspace_unit": "nexu.medium",
20+
"windows_tools": "github.windows",
21+
"js_hot": "nexu.medium",
22+
"ui_hot": "nexu.large",
23+
"ui_p0": "nexu.medium",
24+
"ui_p0_heavy": "nexu.xlarge",
25+
"visual_hot": "nexu.large"
26+
},
27+
"economic": {
28+
"control": "github.ubuntu",
29+
"general_medium": "github.ubuntu",
30+
"workspace_unit": "github.ubuntu",
31+
"windows_tools": "github.windows",
32+
"js_hot": "github.ubuntu",
33+
"ui_hot": "github.ubuntu",
34+
"ui_p0": "github.ubuntu",
35+
"ui_p0_heavy": "github.ubuntu",
36+
"visual_hot": "github.ubuntu"
37+
},
38+
"blacksmith": {
39+
"control": "blacksmith.small",
40+
"general_medium": "blacksmith.small",
41+
"workspace_unit": "blacksmith.small",
42+
"windows_tools": "github.windows",
43+
"js_hot": "blacksmith.small",
44+
"ui_hot": "blacksmith.large",
45+
"ui_p0": "blacksmith.large",
46+
"ui_p0_heavy": "blacksmith.large",
47+
"visual_hot": "blacksmith.large"
48+
}
49+
}
50+
}

.github/config/scopes.json

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
{
2+
"schema": { "version": 1 },
3+
"effects": [
4+
"daemon_tests_required",
5+
"web_tests_required",
6+
"tools_dev_tests_required",
7+
"tools_pack_tests_required",
8+
"ui_critical_validation_required",
9+
"ui_p0_validation_required",
10+
"visual_validation_required",
11+
"workspace_validation_required"
12+
],
13+
"matches": {
14+
"daemon-runtime-definition": {
15+
"prefixes": ["apps/daemon/src/runtimes/defs/"],
16+
"exact": [
17+
"apps/daemon/src/runtimes/capabilities.ts",
18+
"apps/daemon/src/runtimes/local-profiles.ts",
19+
"apps/daemon/src/runtimes/metadata.ts",
20+
"apps/daemon/src/runtimes/registry.ts",
21+
"apps/daemon/tests/runtimes/agent-args.test.ts",
22+
"apps/daemon/tests/runtimes/antigravity-model-lock.test.ts",
23+
"apps/daemon/tests/runtimes/atomcode.test.ts",
24+
"apps/daemon/tests/runtimes/byok-opencode.test.ts",
25+
"apps/daemon/tests/runtimes/chat-run-inactivity-timeout.test.ts",
26+
"apps/daemon/tests/runtimes/claude-resume-args.test.ts",
27+
"apps/daemon/tests/runtimes/codebuddy.test.ts",
28+
"apps/daemon/tests/runtimes/codex-resume-args.test.ts",
29+
"apps/daemon/tests/runtimes/detection-resilience.test.ts",
30+
"apps/daemon/tests/runtimes/opencode-resume-args.test.ts",
31+
"apps/daemon/tests/runtimes/registry-and-args.test.ts",
32+
"apps/daemon/tests/runtimes/trae-cli.test.ts"
33+
]
34+
},
35+
"certain-exempt": {
36+
"prefixes": [".vscode/", ".idea/", "docs/", "apps/landing-page/", ".github/ISSUE_TEMPLATE/"],
37+
"exact": ["LICENSE", ".github/CODEOWNERS"],
38+
"exclude": ["match://daemon-doc"]
39+
},
40+
"daemon-doc": { "exact": ["docs/agent-adapters.md"] },
41+
"packaged-leaf": {
42+
"prefixes": [
43+
"apps/desktop/src/", "apps/desktop/tests/", "apps/packaged/src/", "apps/packaged/tests/",
44+
"tools/pack/src/", "tools/pack/tests/", "tools/pack/resources/"
45+
]
46+
},
47+
"daemon-core": {
48+
"prefixes": ["apps/daemon/src/", "apps/daemon/tests/"],
49+
"exact": ["docs/agent-adapters.md"],
50+
"exclude": ["match://daemon-runtime-definition", "prefix://apps/daemon/src/sidecar/"]
51+
},
52+
"medium-exempt": {
53+
"prefixes": ["nix/"],
54+
"exact": [
55+
".gitignore", ".editorconfig", "flake.nix", "flake.lock",
56+
".github/workflows/landing-page-ci.yml", ".github/workflows/landing-page-staging.yml",
57+
".github/workflows/landing-page-production.yml", ".github/workflows/blog-indexing-on-deploy.yml",
58+
".github/workflows/autofix.atom.yml", ".github/workflows/comment.atom.yml",
59+
".github/workflows/report.atom.yml", ".github/workflows/docker-image.yml", ".github/workflows/nix.yml"
60+
],
61+
"regexes": ["\\.(?:md|mdx|txt)$"],
62+
"exclude": ["match://certain-surface"]
63+
},
64+
"certain-surface": {
65+
"include": ["match://certain-exempt", "match://packaged-leaf", "match://daemon-core"]
66+
},
67+
"workspace-exempt": {
68+
"include": ["match://certain-exempt", "match://medium-exempt", "match://packaged-leaf", "match://daemon-core"]
69+
},
70+
"ui-critical-exempt": {
71+
"include": ["match://certain-exempt", "match://medium-exempt", "match://daemon-core"],
72+
"prefixes": ["apps/desktop/", "apps/packaged/", "tools/pack/"]
73+
}
74+
},
75+
"rules": [
76+
{
77+
"id": "certain-exempt-surface", "match": { "include": ["match://certain-exempt"] },
78+
"effects": [], "confidence": "certain"
79+
},
80+
{
81+
"id": "exempt-surface", "match": { "include": ["match://medium-exempt"] },
82+
"effects": [], "confidence": "medium"
83+
},
84+
{
85+
"id": "certain-daemon-core", "match": { "include": ["match://daemon-core"] },
86+
"effects": ["daemon_tests_required", "ui_critical_validation_required", "ui_p0_validation_required", "workspace_validation_required"],
87+
"confidence": "certain"
88+
},
89+
{
90+
"id": "daemon-sources",
91+
"match": {
92+
"prefixes": ["apps/daemon/", "packages/release/", "packages/contracts/", "packages/platform/", "packages/sidecar/", "packages/sidecar-proto/"],
93+
"exclude": ["match://daemon-core"]
94+
},
95+
"effects": ["daemon_tests_required"], "confidence": "medium"
96+
},
97+
{
98+
"id": "web-sources",
99+
"match": { "prefixes": ["apps/web/", "packages/release/", "packages/components/", "packages/contracts/", "packages/host/", "packages/platform/", "packages/sidecar/", "packages/sidecar-proto/"] },
100+
"effects": ["web_tests_required"], "confidence": "medium"
101+
},
102+
{
103+
"id": "runtime-content",
104+
"match": { "prefixes": ["scripts/", "assets/", "skills/", "prompt-templates/", "design-systems/", "design-templates/", "craft/"] },
105+
"effects": ["daemon_tests_required", "web_tests_required"], "confidence": "medium"
106+
},
107+
{
108+
"id": "script-contract-tests", "match": { "prefixes": ["e2e/tests/scripts/"] },
109+
"effects": ["daemon_tests_required", "web_tests_required"], "confidence": "medium"
110+
},
111+
{
112+
"id": "tools-dev-sources",
113+
"match": { "prefixes": ["tools/dev/", "packages/platform/", "packages/sidecar/", "packages/sidecar-proto/"] },
114+
"effects": ["tools_dev_tests_required"], "confidence": "medium"
115+
},
116+
{
117+
"id": "certain-packaged-leaf-sources", "match": { "include": ["match://packaged-leaf"] },
118+
"effects": ["tools_dev_tests_required", "tools_pack_tests_required", "workspace_validation_required"],
119+
"confidence": "certain"
120+
},
121+
{
122+
"id": "tools-pack-sources",
123+
"match": {
124+
"prefixes": ["tools/pack/", "apps/packaged/", "apps/desktop/", "packages/release/", "packages/components/", "packages/host/", "packages/platform/", "packages/sidecar/", "packages/sidecar-proto/"],
125+
"exclude": ["match://packaged-leaf"]
126+
},
127+
"effects": ["tools_pack_tests_required"], "confidence": "medium"
128+
},
129+
{
130+
"id": "workspace-manifests-and-ci",
131+
"match": {
132+
"exact": ["package.json", "pnpm-lock.yaml", "pnpm-workspace.yaml", ".github/workflows/ci.yml", "e2e/package.json"],
133+
"regexes": ["^apps/[^/]+/package\\.json$", "^packages/[^/]+/package\\.json$", "^tools/[^/]+/package\\.json$"]
134+
},
135+
"effects": ["daemon_tests_required", "web_tests_required", "tools_dev_tests_required", "tools_pack_tests_required"],
136+
"confidence": "medium"
137+
},
138+
{
139+
"id": "ui-p0-surface",
140+
"match": {
141+
"prefixes": ["apps/web/", "apps/daemon/", "packages/release/", "packages/components/", "packages/contracts/", "packages/host/", "packages/platform/", "packages/sidecar/", "packages/sidecar-proto/", "e2e/ui/", "e2e/lib/", "e2e/resources/", "e2e/scripts/", ".github/actions/setup-playwright/", ".github/actions/setup-workspace/"],
142+
"exact": ["e2e/package.json", "e2e/playwright.config.ts", "package.json", "pnpm-lock.yaml", "pnpm-workspace.yaml", ".github/workflows/ci.yml", ".github/workflows/ui-extended-main.yml"],
143+
"exclude": ["match://daemon-core"]
144+
},
145+
"effects": ["ui_p0_validation_required"], "confidence": "medium"
146+
},
147+
{
148+
"id": "visual-surface",
149+
"match": {
150+
"prefixes": ["apps/web/", "e2e/lib/playwright/", ".github/actions/setup-playwright/", ".github/actions/setup-workspace/"],
151+
"exact": ["e2e/package.json", "e2e/playwright.visual.config.ts", "e2e/scripts/playwright.ts", "e2e/scripts/visual-report.ts", "pnpm-lock.yaml", ".github/scripts/handoff.py", ".github/workflows/ci.yml", ".github/workflows/comment.atom.yml", ".github/workflows/report.atom.yml", ".github/workflows/visual-baseline.yml"],
152+
"regexes": ["^e2e/ui/visual-[^/]+\\.test\\.ts$"]
153+
},
154+
"effects": ["visual_validation_required"], "confidence": "medium"
155+
},
156+
{
157+
"id": "ci-control-plane",
158+
"match": {
159+
"prefixes": [".github/config/"],
160+
"exact": [
161+
".github/scripts/scopes.py", ".github/scripts/hash.py", ".github/scripts/runners.py",
162+
".github/scripts/lib/config.py", ".github/scripts/lib/github.py"
163+
]
164+
},
165+
"effects": ["web_tests_required", "workspace_validation_required"], "confidence": "medium"
166+
},
167+
{
168+
"id": "ci-rerun-infra-cancel-surface",
169+
"match": { "exact": [".github/workflows/rerun.atom.yml", ".github/scripts/rerun_infra_cancel.py", "e2e/tests/packaged-smoke-workflow.test.ts"] },
170+
"effects": ["web_tests_required", "workspace_validation_required"], "confidence": "medium"
171+
},
172+
{
173+
"id": "workspace-fallback", "match": { "exclude": ["match://workspace-exempt"] },
174+
"effects": ["workspace_validation_required"], "confidence": "medium"
175+
},
176+
{
177+
"id": "ui-critical-fallback", "match": { "exclude": ["match://ui-critical-exempt"] },
178+
"effects": ["ui_critical_validation_required"], "confidence": "medium"
179+
}
180+
],
181+
"matrices": {
182+
"ui_p0": [
183+
{ "name": "entry-settings", "shard": "entry-settings" },
184+
{ "name": "project-workspace", "shard": "project-workspace" },
185+
{ "name": "project-workspace-editor", "shard": "project-workspace-editor" },
186+
{ "name": "project-collab", "shard": "project-collab" },
187+
{ "name": "project-runtime", "shard": "project-runtime" },
188+
{ "name": "workspace-restoration", "shard": "workspace-restoration" }
189+
],
190+
"visual": [
191+
{ "name": "entry-navigation", "files": "ui/visual-entry.test.ts ui/visual-navigation.test.ts" },
192+
{ "name": "settings-workspace", "files": "ui/visual-settings.test.ts ui/visual-workspace.test.ts" }
193+
]
194+
},
195+
"uiP0Shadow": {
196+
"match": "daemon-runtime-definition",
197+
"matrixNames": ["entry-settings", "project-workspace", "project-collab", "project-runtime"]
198+
}
199+
}

0 commit comments

Comments
 (0)