Milestone 4 — `@hulumi/drift.adapters.GithubWebhookFallbackAdapter` + verdict matrix extension (`tier-degraded`, `feature-not-licensed`) + cache schema migration

Parent runbook: docs/slo/completed/RUNBOOK-hulumi-github.md. Read the runbook's Global Execution Rules (especially Rule 0 — the infra-only scope contract) + Global Entry Rules + lessons files for M1 / M2 / M3 before starting.

Goal: After M4, @hulumi/drift extends with a fifth adapter (GithubWebhookFallbackAdapter) that ingests push-model GitHub webhook events for six event types (branch_protection_rule, repository_ruleset, secret_scanning_alert, dependabot_alert, code_scanning_alert, member, plus org-only organization), verifies HMAC signatures with crypto.timingSafeEqual, deduplicates via an idempotency cache (mode 0600), and feeds drift signals into the existing HardenedVerdict compositor. The DriftVerdict type extends with two first-class fields — tierDegraded?: boolean (true when GitHub plan tier is Team / Pro / Free, i.e. non-GHEC, preventing full audit-log fidelity — distinct from Hulumi's Tier enum which uses values "sandbox" / "startup-hardened") and featureNotLicensed?: string[] (e.g., ["code_scanning_alert"] when the repo isn't GHAS-licensed) — neither of which can be silently suppressed in the verdict output. tierDegraded is set unconditionally by the webhook-fallback adapter (which exists only because the GitHub plan tier denies audit-log API access); it is not a Hulumi-Tier-derived flag. The DriftSource enum gains two values (github-webhook-event, github-product-change) and the cache schema bumps from v1 to v2 with explicit migration logic that preserves all existing AWS-side state.

Context: Research synthesis §"GHEC tier reality" anchors the design: GitHub's enterprise audit-log REST endpoints are GHEC-only and accept only classic PATs — at the wedge persona's Team / Pro tier, the only programmatic drift signal is webhooks. The webhook-events catalog (raw.md §"Webhook fallback feasibility") confirms delivery of the six event types at non-GHEC tiers, with two named carve-outs: org-level events require a separate org-scoped webhook (not just per-repo subscriptions), and private-repo code_scanning_alert requires GHAS so the classifier verdict must distinguish feature-not-licensed from no-drift. The TLA+ spec at docs/TLAdocs/hulumi/HulumiDrift.tla does NOT need re-verification — the GitHub adapter is one of N adapters whose signal participates in the existing HardenedVerdict composition; the verdict-composition rules are unchanged. The cache schema bump v1 → v2 is intentional: forcing explicit acknowledgment of the new adapter prevents stale verdicts from old AWS-only state files appearing as confident "no GitHub drift" outputs. Migration runs on first read; backed up v1 file is preserved at <cache>.v1.backup for one rotation.

Important design rule: tierDegraded: true and featureNotLicensed: string[] are non-suppressible in DriftVerdict output — there is no flag to hide them, and the DriftClassifier.classify() API contract is to surface them whenever they apply. Webhook signature verification uses Node's crypto.timingSafeEqual (constant-time compare) and is mandatory at Hulumi Tier: "startup-hardened" (Hulumi's tier — the governance setting on the adapter, distinct from GitHub plan tier) — Hulumi Tier: "sandbox" may skip signature verification only with an explicit args.allowUnsignedWebhooks: true opt-in, recorded in the audit log. The HMAC secret is sourced from args.webhookSecret: pulumi.Output<string> (e.g., from AWS Secrets Manager or Pulumi Cloud secrets) — never an env var directly, never a string literal in code. **The idempotency cache hashes its keys via crypto.createHash("sha256").update(\${deliveryId}|${eventType}|${repoFullName}`).digest("hex")before any filesystem write** (per critique S5) — rawrepoFullNamefrom the payload is never used as a path component. The cache is bounded by a TTL of 7 days (matching GitHub's Git-event audit-log retention floor) to prevent unbounded growth. **Webhook events arriving out of order are sequenced before composition** (per critique E1): the adapter sorts incoming events by GitHub's server-suppliedX-GitHub-Hook-Installation-Target-ID+received_atenvelope timestamps before submitting them to the verdict compositor; if the timestamps are missing or unparseable, the adapter falls back to ingestion order and emits asecurity_event.webhook_envelope_missing_timestamps` audit row. Webhook payloads are size-bounded at 25 MB (matching GitHub's documented webhook payload limit per GitHub Docs — Creating webhooks; supersedes the earlier 5 MB number per critique S1) AND nesting-depth bounded at 64 levels via a depth-counted recursive-descent parser; payloads exceeding either bound are rejected before any field is read.

Refactor budget: Surgical addition + targeted extension. New packages/drift/src/adapters/github-webhook-fallback.ts, new test files. Targeted extensions to packages/drift/src/types.ts (additive: tierDegraded, featureNotLicensed, two new DriftSource values), packages/drift/src/verdict.ts (the matrix gains rows handling the new sources but the composition logic is unchanged — verified by the existing tla-alignment.test.ts continuing to pass), packages/drift/src/classifier.ts (orchestrate the fifth adapter via Promise.allSettled), and packages/drift/src/cache.ts (schema migration v1 → v2).

Contract Block

Field	Value
Inputs	(Adapter constructor) `new GithubWebhookFallbackAdapter({ webhookSecret: pulumi.Output<string>, idempotencyCachePath: string, tier: Tier, allowUnsignedWebhooks?: boolean })`. (Webhook ingestion) HTTP POST from GitHub with `X-Hub-Signature-256`, `X-GitHub-Delivery`, `X-GitHub-Event` headers + JSON body matching the GitHub webhook event schema. (Classifier integration) `DriftClassifier.classify(urns: string[])` extends to fan out the GitHub adapter alongside the four AWS adapters via `Promise.allSettled`.
Outputs	(Adapter) `DriftSignal { source: "github-webhook-event" \| "github-product-change", confidence: "high" \| "medium" \| "low", evidence: { eventType, deliveryId, repoFullName, occurredAt } }`. (Classifier) `DriftVerdict` extended additively with optional `tierDegraded?: boolean`, `featureNotLicensed?: string[]` fields. Existing `verdict`, `source`, `confidence`, `evidence` fields unchanged for AWS-only paths. (Cache) `v2` schema includes new top-level `githubWebhookCache: Record<deliveryId, { eventType, repoFullName, occurredAt, processed }>` plus `schemaVersion: 2`.
Interfaces touched	New stable surface: `@hulumi/drift#GithubWebhookFallbackAdapter`, `@hulumi/drift#GithubWebhookFallbackAdapterArgs`. Additive extensions to existing `DriftVerdict`, `DriftSource` enum (new values: `github-webhook-event`, `github-product-change`). `DriftAdapter` interface unchanged (the fifth adapter implements the same contract). `DriftClassifier` constructor signature unchanged — adapters list grows. Cache schema migrated from `v1` to `v2`; first-read migration is automatic.
Data classification	`Restricted` — webhook secrets are HMAC keys; their compromise allows attackers to forge GitHub events into the drift cache. Signature verification + idempotency cache + 0600 file permissions are mandatory at startup-hardened. Webhook payloads themselves are `Internal` (event metadata about repo state, no source code, no PII) but the secret is `Restricted` and is handled per `Forbidden shortcuts (a)` below.
Proactive controls in play	(a) `@hulumi/drift` (existing precedent — four AWS adapters, `HardenedVerdict` compositor, TLA+-aligned matrix, cache mode 0600). (b) `C5 Validate All Inputs` — webhook payload schema is validated against expected shape before any field is read; unknown event types are rejected; oversized payloads (>5 MB per GitHub spec) are rejected. (c) `C6 Implement Digital Identity` — HMAC-SHA-256 signature verification using `crypto.timingSafeEqual`. (d) `C7 Enforce Access Controls` — `webhookSecret` is `pulumi.Output<string>` — accessible only to the `pulumi up` IaC role; never written to disk in plaintext, never logged, never `console.log`-ed. (e) `C8 Protect Data Everywhere` — cache file mode 0600; `webhookSecret` value never persisted to cache. (f) `C9 Implement Security Logging and Monitoring` — every signature failure, every replay attempt, every `tierDegraded` verdict, every `featureNotLicensed` event emits a structured `security_event.*` row to stderr. (g) `C10 Handle All Errors and Exceptions` — adapter failures are isolated via `Promise.allSettled`; never crash the classifier; surface as `confidence: "low"` signals with explicit error reason.
Abuse acceptance scenarios	Five BDD rows in the table below cite `tm-hulumi-github-abuse-N`. Slug-keyed: `tm-hulumi-github-abuse-webhook-signature-tampered` (HMAC mismatch rejected via `timingSafeEqual`), `tm-hulumi-github-abuse-webhook-replay-attack` (idempotency cache catches replay via `(deliveryId, eventType, repoFullName)` triple), `tm-hulumi-github-abuse-tier-degraded-not-silent` (`tierDegraded: true` is non-suppressible — no API flag to hide it), `tm-hulumi-github-abuse-feature-not-licensed-honest` (private-repo `code_scanning_alert` without GHAS produces `featureNotLicensed: ["code_scanning_alert"]`, never silent no-drift), `tm-hulumi-github-abuse-cache-migration-no-data-loss` (v1 → v2 migration preserves all AWS-side state and writes a `.v1.backup` file).
Files allowed to change	New: `packages/drift/src/adapters/github-webhook-fallback.ts`; `packages/drift/tests/github/github-webhook-fallback.test.ts`; `packages/drift/tests/github/tier-degraded-verdict.test.ts`; `packages/drift/tests/github/feature-not-licensed-verdict.test.ts`; `packages/drift/tests/github/cache-migration-v1-to-v2.test.ts`; `packages/drift/tests/integration/github/webhook-fallback.integration.test.ts`; `tests/fixtures/webhooks/{branch_protection_rule.json,repository_ruleset.json,secret_scanning_alert.json,dependabot_alert.json,code_scanning_alert.json,member.json,organization.json}` (real-payload-shape examples, redacted of any org/user identifiers); `docs/slo/lessons/hulumi-github-m4.md`; `docs/slo/completion/hulumi-github-m4.md`. Modified: `packages/drift/src/types.ts` (additive: `tierDegraded`, `featureNotLicensed`, two `DriftSource` values, adapter args type); `packages/drift/src/verdict.ts` (matrix rows for the new sources — composition logic unchanged); `packages/drift/src/classifier.ts` (orchestrate fifth adapter); `packages/drift/src/cache.ts` (schema migration v1 → v2 with backup file); `packages/drift/src/index.ts` (re-export adapter); `packages/drift/tests/tla-alignment.test.ts` (extend to confirm new `DriftSource` values map cleanly into the existing 5-row matrix without re-verifying the TLA+ spec); `docs/slo/completed/RUNBOOK-hulumi-github.md` Milestone Tracker. Files outside this milestone's allow-list — REFUSE TO TOUCH including any `packages/baseline/src/`, any `packages/policies/src/`, any `skills/`, any `examples/`, any `docs/TLAdocs/` (the spec is unchanged in M4 — re-verification flag would be raised here if needed but is explicitly NOT).
Files to read before changing anything	`docs/slo/completed/RUNBOOK-hulumi-github.md` (Global Execution Rules + this milestone in full); `docs/slo/lessons/hulumi-github-m{1,2,3}.md`; `docs/slo/research/hulumi-github/synthesis.md` (GHEC tier paragraph); `docs/slo/research/hulumi-github/raw.md` (Audit-log REST auth-mode constraint, Webhook fallback feasibility, Roadmap / change-feed); `packages/drift/src/types.ts`; `packages/drift/src/classifier.ts`; `packages/drift/src/verdict.ts`; `packages/drift/src/cache.ts`; `packages/drift/src/adapters/{automation-api,cloudtrail,git-log,provider-version}.ts` (all four AWS adapters as pattern reference); `packages/drift/tests/tla-alignment.test.ts`; `docs/TLAdocs/hulumi/HulumiDrift-verified.md` (confirm verdict-composition rules are unchanged for M4).
New files allowed	All "New" entries in `Files allowed to change`.
New dependencies allowed	Test-only dev dep: `@octokit/webhooks-types@^7.x` for type-checking webhook payload fixtures (decision deferred from M1). No runtime deps — webhook signature verification uses Node built-ins (`crypto.timingSafeEqual`, `crypto.createHmac`).
Migration allowed	`yes` — cache schema bump v1 → v2 with explicit migration. The migration: (i) reads existing `<cache>.json` with `schemaVersion: 1`; (ii) writes `<cache>.v1.backup` preserving the original; (iii) constructs the v2 shape by adding `githubWebhookCache: {}` and `schemaVersion: 2`; (iv) writes back atomically with mode 0600. If the v1 file is malformed, the migration fails loudly — no silent data loss.
Compatibility commitments	`GithubWebhookFallbackAdapter` and `GithubWebhookFallbackAdapterArgs` are stable from M4. The two new `DriftSource` enum values are locked. The two new `DriftVerdict` fields (`tierDegraded`, `featureNotLicensed`) are optional in shape but non-suppressible in semantics — when conditions warrant, they appear; no consumer flag hides them. Cache schema `v2` is locked; future bumps follow the same migration discipline. Existing AWS adapters and the `HardenedVerdict` composition are unchanged.
Forbidden shortcuts	(a) NEVER log, persist, or surface `webhookSecret` value in stderr, cache, error messages, or any output. The secret is read once on adapter construction, used for HMAC verification, and never written. (b) NEVER skip HMAC verification at Hulumi `Tier: "startup-hardened"`. Hulumi `Tier: "sandbox"` `allowUnsignedWebhooks: true` is the only opt-out, recorded in the audit log. (c) NEVER use `===`, `==`, or string-equality compare for HMAC verification — `crypto.timingSafeEqual` only. (d) NEVER silently fall back to "no GitHub drift" when the adapter cannot reach the cache file or when signature verification fails — emit `confidence: "low"` with an explicit error reason. (e) NEVER suppress `tierDegraded` or `featureNotLicensed` in `DriftVerdict` output. They are non-suppressible by design. (f) NEVER trust `X-GitHub-Delivery` header value as the cache key alone — use the SHA-256 hash of the triple `(deliveryId, eventType, repoFullName)` (per critique S5 — raw `repoFullName` is never a path component). (g) NEVER allow the idempotency cache to grow unbounded — the 7-day TTL is mandatory; eviction runs on every `classifier.classify()` call. (h) NEVER call `child_process.exec`, `eval`, or shell-out anywhere in the adapter or webhook-handling code. (i) NEVER migrate the cache schema without first writing the `.v1.backup` file — atomic write order is backup-first, then v2-write. (j) NEVER parse webhook payloads with the default `JSON.parse` without size + depth bounds (per critique S1) — use a depth-counted parser capped at 64 levels and a body-size cap at 25 MB; reject before any field is read. (k) NEVER ingest events without ordering (per critique E1) — sort by envelope timestamp before composition; on missing timestamps, fall back to ingestion order AND emit `security_event.webhook_envelope_missing_timestamps`. (l) NEVER continue HMAC verification with an old secret without surfacing rotation drift (per critique E3) — when verification fails for >3 consecutive deliveries from the same source, emit `security_event.webhook_secret_rotation_suspected` with remediation guidance.

Out of Scope / Must Not Do

No SLSA-L3 release — that's M5.
No classic-PAT-authed audit-log REST adapter — that's a v1.1 deferral. The runbook's tier decision (Team / Pro) explicitly excludes this work from v1.
No EnterpriseSecurityAnalysisSettings drift detection — GHEC-only, v1.1.
No audit-log streams configuration — v1.1.
No webhook-receiver HTTP server — Hulumi-for-GitHub does not host webhooks; the adapter ingests events from a user-provided file/stream/queue. The cookbook (M5) documents wiring options (AWS Lambda + API Gateway, Cloudflare Worker, etc.).
No DriftAdapter interface change — additive only at the type level.
No re-verification of HulumiDrift.tla — the M4 design rule confirms verdict-composition rules are unchanged. If a future change reshapes composition (e.g. per-source weighting), /slo-tla re-verification becomes required and is flagged at that point.
No suppression of tierDegraded or featureNotLicensed — non-suppressible by design.
No CodeQL queries / Semgrep rules / custom secret-scanning patterns (Rule 0 — infra-only contract).

Pre-Flight

Complete the Global Entry Rules in ../RUNBOOK-hulumi-github.md.
Read docs/slo/lessons/hulumi-github-m1.md, m2.md, m3.md.
Read files listed in Files to read before changing anything. Pay particular attention to packages/drift/tests/tla-alignment.test.ts and the verified-design doc — confirm the assumption that the new DriftSource values fold into the existing matrix without changing composition rules.
Copy the Evidence Log template into the milestone's Evidence Log section.
Re-state in working notes the four load-bearing constraints: (i) HMAC verification is mandatory at startup-hardened; (ii) tierDegraded and featureNotLicensed are non-suppressible in verdict output; (iii) cache schema migration writes .v1.backup before v2-write — atomic order matters; (iv) TLA+ spec is unchanged — if the new adapter reshapes verdict composition, halt and run /slo-tla before continuing.
Confirm webhook-fixture payloads at tests/fixtures/webhooks/ are obtained from real GitHub deliveries (not hand-crafted) and are scrubbed of org/user identifiers.

Files Allowed To Change

File	Planned Change
`packages/drift/src/adapters/github-webhook-fallback.ts`	NEW: `GithubWebhookFallbackAdapter implements DriftAdapter`; HMAC verification, idempotency cache, six event types, `confidence` mapping, `tierDegraded` / `featureNotLicensed` outputs
`packages/drift/src/types.ts`	MODIFY: additive — add `tierDegraded?: boolean` and `featureNotLicensed?: string[]` to `DriftVerdict`; add `"github-webhook-event"` and `"github-product-change"` to `DriftSource` enum; add `GithubWebhookFallbackAdapterArgs`
`packages/drift/src/verdict.ts`	MODIFY: matrix rows for the new sources — composition logic unchanged. Add explicit handling: `tierDegraded` is set when source is `github-webhook-event` AND tier is non-GHEC; `featureNotLicensed` populated by adapter signal
`packages/drift/src/classifier.ts`	MODIFY: include `GithubWebhookFallbackAdapter` in the `Promise.allSettled` fanout when constructed; surface `tierDegraded` and `featureNotLicensed` in the final `DriftVerdict` (no flag to suppress)
`packages/drift/src/cache.ts`	MODIFY: schema migration v1 → v2 with `.v1.backup` file; atomic write; mode 0600 preserved
`packages/drift/src/index.ts`	MODIFY: re-export `GithubWebhookFallbackAdapter`, `GithubWebhookFallbackAdapterArgs`
`packages/drift/tests/tla-alignment.test.ts`	MODIFY: add a meta-assertion that the new `DriftSource` values map into the existing 5-row matrix without changing composition rules; if the assertion fails, the test fails with a pointer to `/slo-tla` re-verification
`packages/drift/tests/github/github-webhook-fallback.test.ts`	NEW: BDD covering signature happy path, signature mismatch, replay attack, unknown event type, oversized payload, six event-type fixtures × happy path, cache TTL eviction
`packages/drift/tests/github/tier-degraded-verdict.test.ts`	NEW: assert `tierDegraded: true` appears in `DriftVerdict` when adapter signals it; non-suppressible — no flag exists to hide it
`packages/drift/tests/github/feature-not-licensed-verdict.test.ts`	NEW: assert `featureNotLicensed: ["code_scanning_alert"]` appears for private-repo `code_scanning_alert` events without GHAS context
`packages/drift/tests/github/cache-migration-v1-to-v2.test.ts`	NEW: seeded v1 cache file → adapter loads → migration runs → `.v1.backup` exists with original content → primary file is v2 shape with `githubWebhookCache: {}` and AWS-side state preserved
`packages/drift/tests/integration/github/webhook-fallback.integration.test.ts`	NEW: real-sandbox-org integration (gated on `HULUMI_INTEGRATION=1` + `HULUMI_GITHUB_SANDBOX_ORG`); creates a webhook on a sandbox repo, triggers a `repository_ruleset.edited` event by toggling a ruleset, asserts the adapter records the drift signal
`tests/fixtures/webhooks/*.json`	NEW: 7 redacted-fixture payloads matching the documented event types
`docs/slo/runbook-milestones/hulumi-github-m4.md`	MODIFY (during execution only): fill Evidence Log rows
`docs/slo/lessons/hulumi-github-m4.md`	NEW (during exit): surprises (especially TLA+-alignment edge cases, cache-migration corner cases), decisions, deltas-from-plan
`docs/slo/completion/hulumi-github-m4.md`	NEW (during exit): changed files, tests added, docs updated
`docs/slo/completed/RUNBOOK-hulumi-github.md` Milestone Tracker	MODIFY: M4 row → `in_progress` on start, `done` on exit

Step-by-Step

Write BDD test stubs covering the five abuse-case rows + happy paths + cache migration + TLA+-alignment meta-test extension. Run — confirm failures for the expected reasons.
Capture seven webhook-fixture payloads from real GitHub deliveries (sandbox org), scrub identifiers, commit to tests/fixtures/webhooks/.
Extend packages/drift/src/types.ts with the additive fields and enum values.
Implement GithubWebhookFallbackAdapter. Constructor validates webhookSecret is pulumi.Output<string> (not a raw string). HMAC verification uses crypto.createHmac("sha256", secret) + crypto.timingSafeEqual(received, computed). Idempotency cache keys events by (deliveryId, eventType, repoFullName). TTL eviction runs on every adapter invocation. The six event types map to drift signals per the documented confidence ladder.
Implement cache migration v1 → v2 in packages/drift/src/cache.ts. Atomic order: read v1 → write .v1.backup → construct v2 → write atomically with mode 0600. Test: feed a malformed v1 file, assert migration fails with a clear message and no data loss.
Extend packages/drift/src/verdict.ts matrix to cover the two new sources. Confirm via tla-alignment.test.ts extension that the new sources fold into the existing 5-row matrix without changing composition rules. If the assertion fails, halt and surface /slo-tla re-verification need.
Wire GithubWebhookFallbackAdapter into DriftClassifier. Promise.allSettled fanout. tierDegraded and featureNotLicensed surfaced in DriftVerdict output unconditionally when adapter signals them.
Run pnpm -r build && pnpm -r test && pnpm -r typecheck && pnpm -r lint && pnpm run lint:license-boundary && pnpm run lint:exact-pin-guard — all green.
Run integration test against sandbox org (manual): confirm webhook event triggers a real drift signal with verified HMAC; teardown removes the webhook.
Smoke tests + Self-Review Gate. Update Milestone Tracker to done, write lessons + completion files (record any TLA+-alignment surprises and cache-migration edge cases).

BDD Acceptance Scenarios

Feature: GithubWebhookFallbackAdapter ingests verified GitHub webhooks and emits drift signals; DriftVerdict carries tierDegraded + featureNotLicensed honestly; cache schema migrates v1 → v2 without data loss

Scenario	Category	Given	When	Then	Threat-model row	Control
Happy path — `repository_ruleset.edited` produces drift signal	happy path	adapter constructed at startup-hardened with valid `webhookSecret`; valid `repository_ruleset.edited` payload with correct `X-Hub-Signature-256`	adapter `ingest(payload, headers)` is called	adapter records the event in idempotency cache; emits `DriftSignal { source: "github-webhook-event", confidence: "high", evidence: { eventType: "repository_ruleset", deliveryId, repoFullName, occurredAt } }`	n/a (happy path)	n/a
Happy path — six event types ingest cleanly	happy path	seven fixture payloads (six event types + org-only `organization`)	each is ingested in turn	each produces a valid `DriftSignal`; no duplicates emitted across the fixtures	n/a	n/a
Empty state — adapter without webhooks	empty state	adapter constructed; no events ingested	`classifier.classify(urns)` runs	adapter contributes `confidence: "low"` "no signal" entry; classifier verdict is unaffected by the empty adapter; AWS-side adapters remain authoritative	n/a	n/a
Dependency failure — cache file unreadable	partial failure	adapter constructed; cache file at `idempotencyCachePath` lacks read permissions	adapter ingests an event	adapter emits `confidence: "low"` with explicit error reason; classifier surfaces this in the final verdict; never silently passes (no false negative)	n/a	explicit error propagation
Abuse case — webhook with tampered signature rejected	abuse case	adapter at startup-hardened with valid `webhookSecret`; payload body is unmodified but `X-Hub-Signature-256` header is altered	adapter `ingest(payload, headers)`	adapter rejects the event; emits `security_event.webhook_signature_failed` to stderr; idempotency cache is NOT updated; constant-time compare via `crypto.timingSafeEqual` (verified via timing-attack test fixture)	`tm-hulumi-github-abuse-webhook-signature-tampered`	`crypto.timingSafeEqual` HMAC verification
Abuse case — webhook replay rejected by idempotency cache	abuse case	adapter; identical `(deliveryId, eventType, repoFullName)` triple ingested twice	second ingestion	adapter emits `security_event.webhook_replay_blocked`; no `DriftSignal` emitted for the second event; classifier verdict reflects only the first ingestion	`tm-hulumi-github-abuse-webhook-replay-attack`	idempotency cache keyed by triple
Abuse case — `tierDegraded: true` is non-suppressible	abuse case	adapter signals `tierDegraded: true` for a Team-tier event	classifier emits `DriftVerdict`	`verdict.tierDegraded === true`; no API flag exists on `DriftClassifier` to suppress the field; type-system attempt to set `tierDegraded: false` post-construction fails (read-only on the verdict shape)	`tm-hulumi-github-abuse-tier-degraded-not-silent`	non-suppressible verdict field; type lock
Abuse case — `featureNotLicensed` honest output	abuse case	adapter receives a private-repo `code_scanning_alert` payload with `repository.private: true` AND no GHAS license context	adapter ingests	`DriftVerdict.featureNotLicensed` includes `"code_scanning_alert"`; `verdict.confidence` is `"low"`; never a silent `no-drift` for a tier-gated feature; consumer can choose to act or ignore but is never misled	`tm-hulumi-github-abuse-feature-not-licensed-honest`	explicit field surfacing; non-silent
Abuse case — cache schema migration v1 → v2 preserves AWS-side state	abuse case	a seeded cache file at `schemaVersion: 1` containing AWS-side cloudtrail + automation-api + git-log + provider-version state	adapter / classifier first-read invocation	`<cache>.v1.backup` exists with byte-for-byte original content; primary cache file is v2 shape; AWS-side state preserved; `githubWebhookCache: {}` initialized; subsequent reads operate on v2 transparently; if v1 file is malformed, migration aborts with explicit error and no data loss	`tm-hulumi-github-abuse-cache-migration-no-data-loss`	atomic write order: backup → v2-write
Schema / compatibility — `DriftVerdict` shape lock	schema / compatibility	`packages/drift/src/types.ts` exports `DriftVerdict`	`tests/skill-bdd/drift-verdict-shape.test.ts` (extended in M4) runs	`DriftVerdict` includes optional `tierDegraded?: boolean` and `featureNotLicensed?: string[]` fields; existing fields unchanged; no v1.x consumer breaks	n/a	type-layer schema lock
TLA+ alignment — new sources fold into existing matrix	schema / compatibility	`packages/drift/tests/tla-alignment.test.ts` (extended in M4)	extended assertion runs	the two new `DriftSource` values map cleanly into the 5-row matrix without changing composition rules; existing AWS-side rows unchanged; if assertion fails, message points at `/slo-tla` re-verification need	n/a	TLA+-alignment meta-test
Abuse case — out-of-order webhook delivery sequenced before composition	abuse case	adapter receives `branch_protection_rule.deleted` AT 14:00:00 envelope, then `branch_protection_rule.created` AT 13:59:58 envelope (reordered by network)	adapter `ingest()` is called in the reordered sequence	adapter sequences by envelope timestamp before submitting to compositor; the verdict reflects the actual final state ("created at 13:59:58, then deleted at 14:00:00 → drift detected at 14:00:00"); never emits a "deleted" verdict that the subsequent re-creation overrides; if envelope timestamps are missing or unparseable, falls back to ingestion order AND emits `security_event.webhook_envelope_missing_timestamps`	`tm-hulumi-github-abuse-webhook-out-of-order-delivery`	envelope-timestamp sequencing pre-compose
Abuse case — webhook secret rotation surfaces structured drift signal	abuse case	adapter constructed with secret `S1`; admin rotates GitHub-side webhook secret to `S2` mid-flight; adapter still uses `S1` (Pulumi state hasn't refreshed)	three webhook deliveries arrive signed with `S2`; HMAC verification fails for all three	after the third consecutive failure from the same `(installation_id, repo_full_name)` source, adapter emits `security_event.webhook_secret_rotation_suspected` with `{ installation_id, repo_full_name, consecutive_failures: 3, remediation: "run pulumi up to refresh webhookSecret" }`; subsequent deliveries continue to fail-then-warn until secret is refreshed; the failure path never silently degrades to `no-drift`	`tm-hulumi-github-abuse-webhook-secret-rotation`	rotation-detection counter + structured audit row
Abuse case — deeply-nested payload rejected before parse	abuse case	webhook payload at 23 MB containing 5,000 levels of nested JSON `{"a":{"a":{...}}}`	adapter `ingest(payload, headers)` is called	depth-counted recursive-descent parser rejects at level 65 (depth limit 64); error `payload_max_nesting_depth_exceeded` emitted; idempotency cache is NOT updated; classifier verdict surfaces `confidence: "low"` with reason; no V8 stack overflow, no unbounded memory	`tm-hulumi-github-abuse-payload-deserialization-bomb`	depth-counted parser + 64-level cap
Abuse case — oversized payload rejected before parse	abuse case	webhook payload at 26 MB (above the 25 MB GitHub spec)	adapter `ingest()`	size check runs before parse; rejects with `payload_max_size_exceeded`; idempotency cache is NOT updated; verdict reflects the rejection	`tm-hulumi-github-abuse-payload-deserialization-bomb`	25 MB body-size cap pre-parse
Abuse case — path-traversal in repository.full_name neutralized at cache key	abuse case	webhook payload with `repository.full_name: "../../../tmp/attacker-controlled"` (only reachable if HMAC is also bypassed; defense-in-depth)	adapter ingests; cache write attempted	the cache writes to `<cache-dir>/<sha256-of-triple>.json`, never `<cache-dir>/../../../tmp/attacker-controlled-<deliveryId>.json`; assertion: every cache filename matches `[0-9a-f]{64}\.json` regex; no filesystem write outside the configured cache directory under any payload input	`tm-hulumi-github-abuse-cache-key-path-traversal`	SHA-256-hashed cache key + filename regex assertion

Regression Tests

All M1, M2, M3 BDD scenarios pass; in particular hulumi:controls tag (M3) is unchanged on M4 outputs.
All AWS BDD scenarios continue to pass; the four AWS adapters are unmodified.
packages/drift/tests/tla-alignment.test.ts continues to pass — extended in M4 to cover the new sources without re-verifying the spec.
pnpm run lint:license-boundary continues to pass.
pnpm run lint:exact-pin-guard continues to pass.

Compatibility Checklist

E2E Runtime Validation

E2E Test	What It Proves	Pass Criteria
`webhook_signature_verification_constant_time`	HMAC verification resists timing attacks	timing-attack test (50 iterations comparing valid vs invalid signatures) shows < 5% delta in mean time; `crypto.timingSafeEqual` is documented to be constant-time at the std-lib level
`replay_attack_blocked_across_repos`	Cross-repo replay defended by triple-keyed idempotency	seeded `(deliveryId, eventType, repoFullName)` triples replayed with different `repoFullName` values are NOT blocked; replays of identical triples ARE blocked
`tier_degraded_appears_for_team_tier_events`	Tier-gating honesty	adapter at Team tier produces `DriftVerdict.tierDegraded === true` for any drift signal; no flag in `DriftClassifier` API hides it
`feature_not_licensed_for_private_code_scanning_without_ghas`	Feature-gating honesty	private-repo `code_scanning_alert` payload without GHAS context produces `DriftVerdict.featureNotLicensed === ["code_scanning_alert"]`; never silent no-drift
`cache_migration_v1_to_v2_preserves_aws_state_atomically`	No data loss on schema bump	seeded v1 file containing AWS-side state → migration → `.v1.backup` exists with original bytes → v2 file has all original AWS keys preserved + `githubWebhookCache: {}` + `schemaVersion: 2`
`tla_alignment_meta_test_extends_to_new_sources`	New sources don't reshape verdict composition	extended assertion passes; if it fails, error message names `/slo-tla` re-verification
`real_sandbox_webhook_creates_drift_signal`	Integration with real GitHub webhook	sandbox-org webhook on `repository_ruleset` event triggered by toggling a ruleset → adapter records signal → classifier emits `DriftVerdict` with `source: "github-webhook-event"`; teardown removes webhook
`out_of_order_webhook_sequencing`	E1 — envelope-timestamp sequencing	reordered fixture pair (`deleted@14:00:00`, then `created@13:59:58`) submitted; verdict reflects actual final state, not ingestion order; missing-timestamp fixture emits `security_event.webhook_envelope_missing_timestamps`
`webhook_secret_rotation_emits_structured_audit_row`	E3 — rotation-detection	three consecutive HMAC failures from the same `(installation_id, repo_full_name)` source emit `security_event.webhook_secret_rotation_suspected` with remediation guidance; one-off failures from disparate sources do not trigger the rotation event
`deeply_nested_payload_rejected_before_parse`	S1 — depth bound	5,000-level nested JSON payload rejected at level 65; no V8 stack overflow; cache untouched
`oversized_payload_rejected_before_parse`	S1 — size bound	26 MB payload rejected at size check; cache untouched
`cache_key_sha256_hashed_no_path_traversal`	S5 — path-traversal neutralization	for any payload input, every cache filename matches `[0-9a-f]{64}\.json`; no write outside configured cache directory

Smoke Tests

pnpm install --frozen-lockfile && pnpm -r build && pnpm -r test && pnpm -r typecheck && pnpm -r lint && pnpm run lint:license-boundary && pnpm run lint:exact-pin-guard → all green.
(Requires creds) HULUMI_INTEGRATION=1 HULUMI_GITHUB_SANDBOX_ORG=<org> HULUMI_GITHUB_APP_ID=... pnpm test:integration → integration test green; sandbox org has no leaked webhooks after the run.
Cache file at ~/.hulumi/drift-cache/<run-id>.json shows schemaVersion: 2; ~/.hulumi/drift-cache/<run-id>.v1.backup exists if upgrading from v1.
git status clean; .gitignore covers any new generated files (incl. test fixtures and migration backups).

Evidence Log

Step	Command / Check	Expected Result	Actual Result	Pass/Fail	Notes
Baseline tests
BDD test stubs created
Webhook fixtures captured
Types extended
Adapter implemented
Cache migration implemented
Verdict matrix extended
Classifier wired
TLA+ alignment meta-test
Build / typecheck / lint
Mock-runtime BDD
Cache migration test
HMAC timing test
Sandbox-org integration
Regression tests
Test artifact cleanup
.gitignore review

Definition of Done

All BDD scenarios pass.
All E2E runtime validation tests pass.
pnpm -r test green; lint + typecheck + license-boundary + exact-pin-guard green.
Smoke tests checked off.
Compatibility checklist complete.
No forbidden shortcuts present.
Cache schema migration tested + .v1.backup mechanism verified.
HMAC verification timing-attack-resistant (constant-time-compare confirmed).
tierDegraded and featureNotLicensed non-suppressible (no API flag hides them).
TLA+ alignment meta-test passes — verdict composition unchanged.
All M1, M2, M3 BDD scenarios pass.
All AWS Hulumi v1.0.0 BDD scenarios pass.
git status clean.
docs/slo/lessons/hulumi-github-m4.md written (TLA+ corner cases + cache migration surprises captured).
docs/slo/completion/hulumi-github-m4.md written.
Milestone Tracker in docs/slo/completed/RUNBOOK-hulumi-github.md updated to done.

Post-Flight

docs/slo/completed/RUNBOOK-hulumi-github.md Milestone Tracker → M4 done.
docs/slo/completed/RUNBOOK-hulumi-github.md Component Summary Table — verify M4 row.
docs/components/github-webhook-fallback-adapter.md — one-line stub.
docs/cookbooks/ — flag M5 to write the "wiring webhooks into Hulumi-for-GitHub drift" cookbook.

Notes

This milestone delivers the most operationally complex single component in the runbook. HMAC verification, idempotency cache, schema migration, and TLA+-alignment are four independent invariants that all must hold.
The cache schema bump v1 → v2 is the most user-facing breaking-feeling change in the runbook (even though the migration is automatic). Lessons file MUST capture user-experience observations from the manual smoke test — does the .v1.backup mechanism feel safe? Is the migration message clear?
The tierDegraded / featureNotLicensed non-suppressibility is a load-bearing UX decision: the runbook recommendation file at docs/slo/research/hulumi-github/synthesis.md anchors the wedge persona's trust in honesty about the tier gap. Hiding these fields would defeat the recommendation.
v1.1 follow-up: classic-PAT-authed audit-log REST adapter for GHEC customers. Track in docs/issue-candidates.md.

FilesExpand file tree

hulumi-github-m4.md

Latest commit

History