|
5 | 5 |
|
6 | 6 | --- |
7 | 7 |
|
| 8 | +### 2026-07-10 — v0.1.25.47: TENANT_CLOSED Rule 2 guard on reservation mutations |
| 9 | + |
| 10 | +Adopts the governance spec's CASCADE SEMANTICS Rule 2 (terminal-owner |
| 11 | +mutation guard, `cycles-governance-admin-v0.1.25.yaml`) on the runtime |
| 12 | +plane: once the owning tenant's CLOSED flip is durable, reservation |
| 13 | +create/commit/release/extend return 409 `TENANT_CLOSED` (Mode B |
| 14 | +invariant (a): a mutation observed after the flip MUST NOT succeed, even |
| 15 | +before the cascade touches the child or revokes keys). |
| 16 | + |
| 17 | +What the guard actually closes, layer by layer (audited during |
| 18 | +implementation): `ApiKeyRepository.validate` ALREADY reads `tenant:<id>` |
| 19 | +fresh per request and 401s tenant keys of SUSPENDED/CLOSED tenants at |
| 20 | +the auth filter — so for tenant-key HTTP traffic the post-flip window |
| 21 | +was already shut (with 401, the pending runtime spec revision's "a |
| 22 | +closed tenant usually surfaces on this plane as 401", not the Rule 2 |
| 23 | +409). Two real gaps remained, both now closed: (1) **admin-key |
| 24 | +mutations** — the runtime's admin-on-behalf-of auth (`X-Admin-API-Key`; |
| 25 | +allowlist GET list/single + POST release) carries no tenant-status |
| 26 | +check, so an admin release on a closed tenant SUCCEEDED, mutating |
| 27 | +budgets post-flip; it now returns 409 `TENANT_CLOSED`. (2) **the |
| 28 | +auth-check→script race** — the filter reads tenant status at auth time |
| 29 | +and the CLOSED flip can land between that read and the Lua execution; |
| 30 | +only an in-script check is atomic with the mutation. The guard also |
| 31 | +covers any future path that reaches the repository without the |
| 32 | +tenant-key filter. |
| 33 | + |
| 34 | +Design decisions: |
| 35 | +- **Guard placement — inside the Lua scripts.** The codebase's precedent |
| 36 | + for status guards (BUDGET_FROZEN/BUDGET_CLOSED in reserve.lua) is |
| 37 | + in-script, and only in-script placement is atomic with the budget |
| 38 | + mutations (Redis executes scripts serially): a Java pre-check would |
| 39 | + leave a flip-vs-mutation race, and piggybacking on |
| 40 | + `getTenantConfig()` would inherit its 60 s Caffeine cache |
| 41 | + (`cycles.tenant-config.cache-ttl-ms`), violating invariant (a)'s |
| 42 | + "durable to readers" requirement. Cost: one extra Redis `GET |
| 43 | + tenant:<id>` + `cjson.decode` inside each mutation script — no extra |
| 44 | + network round-trip. |
| 45 | +- **Owning-tenant resolution.** reserve.lua uses the auth-derived tenant |
| 46 | + already in ARGV[10]; commit/release/extend read the `tenant` field from |
| 47 | + the reservation hash (the authoritative owner; reserve.lua has always |
| 48 | + written it) — added to the scripts' existing HMGETs, no new commands. |
| 49 | +- **No tenant record ⇒ no restriction** (runtime-only deployments), same |
| 50 | + contract as the admin plane's `TerminalOwnerMutationGuard`. A PRESENT |
| 51 | + record that cannot be decoded into an object with a string `status` |
| 52 | + **fails closed** (500 INTERNAL_ERROR, no mutation) — matching the admin |
| 53 | + plane's TenantRepository, which propagates parse failures rather than |
| 54 | + treating a corrupt governance record as an open tenant (codex round 2; |
| 55 | + the first cut fell through open on decode failure). Round 3 extended |
| 56 | + fail-closed to unknown status STRINGS via a whitelist (CLOSED → 409; |
| 57 | + ACTIVE/SUSPENDED → proceed; anything else → 500): the governance |
| 58 | + TenantStatus enum is a closed set and the cascade revision explicitly |
| 59 | + introduces no new status values as a wire-compat guarantee, so an |
| 60 | + unknown status (e.g. "CLOZED", lowercase "closed") cannot be a |
| 61 | + legitimate future value under the current contract — it is corruption. |
| 62 | + Pinned per op with "CLOZED" and lowercase "closed" records in the |
| 63 | + malformed matrix. |
| 64 | +- **Precedence.** Same-key idempotent replay first (a replay re-observes |
| 65 | + a pre-flip mutation — not a new mutation; Rule 2(b) idempotency; |
| 66 | + mirrors how the budget status guards sit after replay), then the |
| 67 | + closed-tenant guard, then every reservation-state/expiry/budget check — |
| 68 | + honoring the spec's "regardless of that child's own current status" |
| 69 | + (precedence sentence added to spec PR runcycles/cycles-protocol#125 |
| 70 | + ERROR SEMANTICS). Codex round 2 resolved the first cut's accepted edge: |
| 71 | + commit.lua/release.lua previously returned RESERVATION_FINALIZED for a |
| 72 | + different-key attempt on a finalized reservation before the guard ran; |
| 73 | + the replay branches were narrowed to true same-key replays so |
| 74 | + different-key attempts fall through to the guard, and release.lua's |
| 75 | + post-guard state check widened from `== "COMMITTED"` to `~= "ACTIVE"` |
| 76 | + to keep the open-tenant RESERVATION_FINALIZED response identical. |
| 77 | +- **Scope.** Exactly the four reservation mutations Rule 2 names get the |
| 78 | + 409 guard. `GET`/list stay available on closed tenants (spec: |
| 79 | + post-mortem reads). The non-persisting evaluations (dry_run + |
| 80 | + `/v1/decide`) were initially left unguarded; an external review round |
| 81 | + (round 4) flagged that a post-flip dry_run could stamp a SIGNED ALLOW |
| 82 | + attestation for a request whose live execution MUST fail — resolved |
| 83 | + per the amended spec PR runcycles/cycles-protocol#125: a FRESH |
| 84 | + evaluation on a CLOSED tenant now returns 200 decision=DENY with |
| 85 | + reason_code=TENANT_CLOSED (new `Enums.ReasonCode` value, typed, |
| 86 | + mirroring the documented DecisionReasonCode vocabulary) via a single |
| 87 | + shared gate (`evaluateTenantStatusGate`) called from both evaluation |
| 88 | + paths, after replay handling and before any budget read. The gate |
| 89 | + reads `tenant:<id>` fresh (never the 60 s config cache) and applies |
| 90 | + the same fail-closed whitelist as the Lua guards — malformed record |
| 91 | + (undecodable / non-object / missing or non-string status / unknown |
| 92 | + status string) → 500 INTERNAL_ERROR BEFORE evidence stamping (the |
| 93 | + server cannot attest against corrupt governance state; no |
| 94 | + reserve/decide evidence row and no error-evidence row is written, |
| 95 | + consistent with the existing convention that evidence is emitted only |
| 96 | + for decisions actually reached — INTERNAL_ERROR is likewise excluded |
| 97 | + from EVIDENCE_DENIAL_CODES). Cached pre-close replays keep their |
| 98 | + original payload. `POST /v1/events` also mutates budgets and Rule 2's list is |
| 99 | + "non-exhaustive" — flagged as an open spec question rather than guarded |
| 100 | + ahead of the spec. `TENANT_CLOSED` error-evidence emission was initially |
| 101 | + deferred "until spec v0.1.25.13 lands"; resolved in review round 5 — |
| 102 | + spec PR runcycles/cycles-protocol#125 added TENANT_CLOSED to the |
| 103 | + evidence ErrorResponseMirror (cycles-evidence-v0.2.yaml 0.2.1) and both |
| 104 | + PRs merge together, so `TENANT_CLOSED` is now IN |
| 105 | + `EVIDENCE_DENIAL_CODES`. Rationale: the set's criterion is "decision |
| 106 | + reached and denied" and it already contains the governance-state |
| 107 | + denials BUDGET_FROZEN/BUDGET_CLOSED; a mutation-surface 409 |
| 108 | + TENANT_CLOSED is the direct sibling of BUDGET_CLOSED (owner-level |
| 109 | + instead of ledger-level), so excluding it was inconsistent — the signed |
| 110 | + denial receipt is exactly what a closed-tenant enforcement event should |
| 111 | + produce. Error-evidence emission applies to the mutation-surface 409s |
| 112 | + (persisting create, commit, release; reservation_id hoisted on |
| 113 | + commit/release). /v1/decide never 409s for a closed tenant — it (and |
| 114 | + dry_run create) returns 200 DENY/TENANT_CLOSED and emits its normal |
| 115 | + decide/reserve decision evidence via the round-4 gate; extend is not |
| 116 | + an evidence endpoint and emits nothing, same as every other code |
| 117 | + (pinned by test). SUSPENDED tenants: existing runtime semantics |
| 118 | + live in the AUTH layer only (tenant keys 401 — pre-existing, unchanged, |
| 119 | + pinned); the mutation-layer guard is deliberately CLOSED-only per |
| 120 | + Rule 2 / spec v0.1.25.13. |
| 121 | +- **Wire.** `Enums.ErrorCode` gains `TENANT_CLOSED` (additive; runtime |
| 122 | + spec revision v0.1.25.13, runcycles/cycles-protocol#125 — mirrors the pre-existing |
| 123 | + governance code). |
| 124 | + |
| 125 | +Tests: `TenantClosedGuardIntegrationTest` (Testcontainers Redis, real |
| 126 | +Lua). All four ops are exercised at the repository layer — a repository |
| 127 | +call is exactly "a request already past auth", i.e. the |
| 128 | +filter-check→script race — with no-partial-mutation assertions (budget |
| 129 | +`reserved`/`remaining`/`spent`, reservation state, `expires_at` |
| 130 | +unchanged). HTTP layer: admin-key release on a closed tenant → 409 |
| 131 | +TENANT_CLOSED with the full ErrorResponse envelope (previously 200 — |
| 132 | +the reachable hole); tenant-key mutation on a closed tenant → 401 |
| 133 | +pinned (pre-existing auth behavior, unchanged); admin GET + list on a |
| 134 | +closed tenant → 200 (Rule 2 read access). Plus record-absent / ACTIVE |
| 135 | +pass-through, SUSPENDED (repo-level ops proceed — mutation guard is |
| 136 | +CLOSED-only; auth-layer 401 pinned as pre-existing), cross-tenant |
| 137 | +isolation, and replay-across-the-flip semantics. The 409 HTTP calls use |
| 138 | +a non-validating client until spec v0.1.25.13 merges (the shared |
| 139 | +validating client checks response enums against cycles-protocol@main); |
| 140 | +response shape is asserted explicitly. Unit tests: handleScriptError |
| 141 | +token mapping, `tenantClosed` factory, GlobalExceptionHandler 409 |
| 142 | +envelope (+ no-evidence pin). |
| 143 | + |
| 144 | +Codex review round 2 (both applied against real-Lua tests): (1) |
| 145 | +fail-closed on malformed tenant records — all four guards return |
| 146 | +INTERNAL_ERROR (500, message preserved through a new explicit |
| 147 | +handleScriptError case) when a present `tenant:<id>` row fails |
| 148 | +cjson.decode, decodes to a non-object, or lacks a string `status`; |
| 149 | +pinned per op with malformed / non-object (string, number) / |
| 150 | +missing-status shapes and no-partial-mutation assertions. (2) |
| 151 | +TENANT_CLOSED precedence over RESERVATION_FINALIZED for non-replay |
| 152 | +mutations (see the Precedence bullet); pinned: closed tenant + |
| 153 | +finalized reservation + different key → 409 TENANT_CLOSED on commit, |
| 154 | +release, and extend; same-key replay still returns the original |
| 155 | +response; open tenant + different key still RESERVATION_FINALIZED |
| 156 | +(no-regression pin). Codex also confirmed the matcher port is |
| 157 | +byte-identical to admin main and that the in-script `GET tenant:<id>` |
| 158 | +matches the repo's existing standalone-Redis posture — no changes. |
| 159 | + |
| 160 | +### 2026-07-10 — v0.1.25.47: webhook scope-filter matcher parity with the admin plane |
| 161 | + |
| 162 | +The admin server fixed its `scope_filter` matcher for spec conformance |
| 163 | +(cycles-server-admin PR #206); the runtime dispatch matcher |
| 164 | +(`EventEmitterRepository.matchesScope`) already had the trailing-`*`/exact |
| 165 | +split but lacked two of the admin refinements, so the two planes could |
| 166 | +disagree on the same (filter, scope) pair — a subscription could receive an |
| 167 | +event live (runtime dispatch) that admin-plane replay would skip, or vice |
| 168 | +versa. Ported both refinements 1:1 so the matchers are byte-identical: |
| 169 | +(1) blank/whitespace-only event scope is unscoped — excluded from any |
| 170 | +scope-filtered subscription (previously bare `*`, an empty-prefix |
| 171 | +`startsWith`, matched a blank `""` scope); (2) trailing-`*` filters require |
| 172 | +a non-empty child segment after the prefix (`tenant:a/*` no longer matches |
| 173 | +the degenerate `tenant:a/`; spec text is "all scopes *under*" the base). |
| 174 | +The matcher was made `public static` (mirroring the admin's) and the |
| 175 | +admin's full matcher test table — null/blank filters, bare `*`, trailing |
| 176 | +`*` child/base/sibling/empty-segment cases, exact match, literal |
| 177 | +mid-string `*`, case sensitivity, blank-scope edges — was ported into |
| 178 | +`EventEmitterRepositoryTest`, pinning both planes to the same |
| 179 | +(filter, scope, expected) table. One dispatch-level test pins the |
| 180 | +blank-scope refinement end-to-end through `emit()`. No wire or storage |
| 181 | +change; delivery selection shifts only on the two edge cases. |
| 182 | + |
8 | 183 | ### 2026-07-04 — full-stack prod compose: stop host-publishing events management port (no version bump) |
9 | 184 |
|
10 | 185 | `docker-compose.full-stack.prod.yml` published the events worker's 9980 to |
|
0 commit comments