Skip to content

0.5.12-plan.md

Latest commit

 

History

History
231 lines (180 loc) · 12.1 KB

0.5.12-plan.md

File metadata and controls

231 lines (180 loc) · 12.1 KB
Raw

Airlock 0.5.12 — Plan

Theme: the data-provenance / egress-control milestone.

One boundary, both directions. 0.5.12's headline is PII rehydration done safely — Airlock reverses its own redaction to make tool calls work, and it must do so without building a PII goldmine or an unaudited egress path. Its companion is the Phase-B indirect-injection / multi-turn input boundary, deferred here from 0.5.11 as a design-first pack. These are the same seam seen from two sides:

  • Egress provenance (PII rehydration): real PII values flow out into tool-call arguments. Which sink may receive them is an egress-control decision.
  • Ingress provenance (injection boundary): untrusted text flows in from tool outputs and prior turns. Which text is trusted at which boundary is the dual question.

Building the provenance concept once, for both directions, is why they belong in one milestone. It is also the strongest form of the GTM "prove control over your organization's data" positioning (dev/notes/gtm-buyers-gatekeepers-overview.md, 0.6.0 tree): not "we have guardrails" but "we can prove where your data went and what reached your model."

Status: implementation complete for the startup/OOM resilience work, A-1 through A-4, and B-0. A-5 is intentionally pending a deployed dogfood window and recorded human DECIDE; it cannot be completed from unit tests or an empty telemetry population. The code commits are e10c842, 9d0e2c2, c9ca67c, e281ec6, and 9579716. pyproject remains 0.5.10 until publication closes.

Publication (owner decision, at closeout): 0.5.12 is intended as the next published release. Publishing it publishes everything since 0.5.10 — the internal 0.5.11 FathomDB work and 0.5.12 — so the version bump is 0.5.10 → 0.5.12 and the CHANGELOG carries both trains (as 0.5.10 did for 0.5.8→0.5.10). Whether to publish at all remains the owner's call at closeout.

Design inputs

The PII design is already done and consolidated, and committed to this branch alongside this plan (relocated from the 0.6.0 worktree where it was drafted):

  • dev/notes/design-pii-rehydration-primary.md — authoritative; the reconciled design.
  • dev/notes/design-pii-rehydration-goldmine.md — Part-A detail (verified threat model, file:line; opaque-handle; graceful-degrade).
  • dev/notes/design-rehydration-authorization-maintainability.md — Part-B detail (research survey; layered egress gate; Cedar/CaMeL/blocklist evidence).

Still DRAFT / owner-review-pending. The injection-boundary design does not exist yet — pack B-0 produces it.

Scope decisions already made

Decision Consequence
PII containment is pack 1, folded into the milestone (owner decision) — not a separate hotfix The live map-in-logs leak ships fixed in the first pack; the on-disk logs/airlock-2026-08-04.jsonl (cleartext maps) is purged as an ops action within that pack
Unified provenance milestone (owner decision) — PII and the injection boundary TUI backlog (#25/#26/#30/#31/#32) is deferred to 0.5.14; this milestone is not TUI ballast
Global-first lists Per-tenant PII allow/block lists are 0.6.0 territory (tenant identity); 0.5.12 ships one global policy, per-tenant-ready in shape
Streaming rehydration stays deferred The non-streaming tool-call path is the deliverable; streaming inherits the invariants when built (own later pack)
Cedar / CaMeL are later tiers v1 needs no external PDP and no dual-LLM rebuild; adopt only when their triggers fire (primary note §4)
Enforce is data-gated The PII egress gate ships in observe mode; flipping to enforce is a HITL DECIDE on measured metrics, not a release-date decision

Release resilience — startup recovery incident (required) · S

On 2026-08-09, production was OOM-killed and systemd correctly attempted to restart it. Recovery instead entered a restart storm: Airlock's GET /health replacement only inspected top-level FastAPI routes, while the locked FastAPI 0.141.1 keeps LiteLLM's route inside an included router. Airlock therefore raised its safety error even though the inherited, provider-fan-out route was present.

  • Find and remove GET /health from both top-level and included FastAPI routers, invalidate affected router caches, then register the safe aggregate route. Do not weaken the fatal check: an actually unreplaceable inherited route must still prevent startup rather than expose a billed liveness path.
  • Add a regression test using FastAPI.include_router() and preserve the structural no-model-call coverage for every Airlock health endpoint.
  • Bound systemd restart storms with StartLimitIntervalSec=5min and StartLimitBurst=3 while retaining automatic retry for transient failures.
  • Done when: the health suite passes against the locked FastAPI/LiteLLM environment, an isolated proxy completes startup, and the deployed service is verified with /livez (never GET /health as a liveness probe).

The OOM trigger remains a separate operational investigation; raising the memory limit alone is not an accepted fix.

Workstream A — PII rehydration done safely (headline)

From design-pii-rehydration-primary.md. Two axes kept distinct: stages = no-goldmine data handling; layers = egress authorization.

A-1 — Containment (pack 1) · S

The live leak, fixed first. Invariant: airlock_pii_map MUST NOT reach any sink or serialized record.

  • Denylist the map at the event-builder snapshot (request_event.py:176) and in write_precall_block_record — no current or future sink can serialize it.
  • Ops action in this pack: purge or scrub logs/airlock-2026-08-04.jsonl (and any other map-bearing log) — jq 'del(.airlock_pii_map)'. A code fix cannot clean data already on disk. (Owner runs this; the file's provenance — real vs. test canary PII — is unconfirmed, so treat as real.)
  • Done when: the T1/T2/T4 canary sweep passes (map + a canary PII string absent from every serialized sink and block record); the on-disk log is clean.

A-2 — No goldmine: opaque handle, lifetime, copy-response · M

  • Opaque handle (A1 stage): the map lives in a bounded, request-scoped, process-local store keyed by a random handle; only the handle rides metadata; deleted in finally on success / exception / timeout / disconnect; TTL sweep for abandoned handles; store is bounded. Closes the LiteLLM-debug-dump surface the denylist cannot.
  • Bounded lifetime + ops rule (A2 stage): no LiteLLM detailed-debug with PII on; ulimit -c 0 on the prod host.
  • Hydrate a copy (A3 stage): telemetry sees the pre-hydration (redacted) response; only the client sees the hydrated one — stop mutating the shared response object in place.
  • Done when: T3/T5/T7/T8 pass (handle not map on the bus; store cleanup on every exit path + TTL + bounded; telemetry object stays redacted).

A-3 — The egress-authorization gate (observe mode) · L

The layered, default-deny gate — built mode-aware and landing in observe.

  • Evaluation order (primary note §3.1): known-bad blocklist (deny-override) → coarse taint → type-compatibility (asymmetric) → sink egress-trust band (round-trip auto-allow / exfil / unknown⇒deny) → residual allowlist.
  • Required known-bad blocklist as the step-0 deny-override (distinct from the rejected allow-by-default fallback): globally-dangerous sinks, incident response, class vetoes. Default-deny handles unknown-bad; the blocklist handles known-bad; neither is exhaustive.
  • Suppressed-hydration telemetry (value-free): every suppression emits an audit event that auto-populates the residual list (learn-then-enforce).
  • v1 builds Layer 2 on the existing mcp_tool_guard allowlist as the trust catalog, extended with an egress band. Layer 1 type-compatibility lands when MCP schemas are available (refinement).
  • Done when: T9 passes (mismatch→suppress; round-trip match→allow; exfil→deny-unless-listed; known-bad blocklist vetoes an otherwise-allowed sink; suppression events are value-free); the gate runs in observe without changing behavior (T11).

A-4 — Graceful-degrade policy · XS

  • AIRLOCK_PII_FAIL_MODE = open | closed for the redaction-unavailable case (default open per directive; regulated deployments set closed), with a loud startup warning and a per-request airlock_pii_unavailable audit marker that survives to the audit sink. Rehydration-unavailable always degrades to the placeholder (safe, no egress).
  • Done when: T10 passes (open serves + stamps the marker; closed blocks).

A-5 — Validation run → DECIDE (enforce vs. blocklist-interim) · M

The falsifiable step. Run the gate observe → shadow → enforce in dogfood with synthetic + canary PII (prod PII is off — isolated). Six pre-registered metrics (primary note §5.2): zero-entry rate, false-deny on canary-legit, residual growth curve, false-allow into exfil, canary egress, unknown-tool rate. HITL DECIDE: enforce the layered allowlist if the criteria hold; fall back to the blocklist-interim only if the zero-entry claim fails for the real tool mix. The known-bad blocklist + telemetry run in every mode regardless.

  • Done when: the metrics are collected, the DECIDE is recorded, and 0.5.12 ships in the decided mode (observe or enforce) — never enforce on faith.

Implementation status: the value-free event field and pii-egress mode of scripts/measurement-report.py are ready. The required traffic matrix and pre-registered thresholds are in dev/plans/runs/0.5.12-pii-egress-measurement-protocol.md. The live Airlock process predates this branch and must be deployed before it can emit the field; do not treat its existing logs as an observe window.

Workstream B — Ingress provenance: indirect-injection / multi-turn boundary

Design-first, deferred here from 0.5.11 ("what text is untrusted, and at which boundary — building that provenance seam twice is the waste").

B-0 — Design + exploration (design-first) · M

  • Define the untrusted-text boundary: tool outputs and prior-turn content are untrusted; where does Airlock mark and carry that provenance through the request lifecycle? How does it compose with the semantic/injection classifiers already shipped (0.5.9) and with the PII egress gate (shared provenance concept — do not build a second provenance mechanism)?
  • Produce a design memo + a DECIDE on the enforcement seam before any IMPL pack.
  • Done when: the design memo exists, is reviewed, and a HITL DECIDE fixes the boundary + enforcement seam. No IMPL pack is authored before this.

Implementation status: design memo is complete at dev/notes/design-indirect-injection-provenance.md. Semantic-classifier selection/review and the enforcement contract remain explicit human gates before any B-1 implementation.

B-1..n — IMPL (authored only after B-0 DECIDE)

Shape TBD by B-0. Provisional: mark untrusted spans at the ingress boundary; carry provenance on the RequestEvent; let the injection classifiers weight untrusted spans; enforce per the observe/shadow/enforce lifecycle.

Explicitly out of scope

  • Streaming PII rehydration — deferred; inherits every invariant when built.
  • Per-tenant PII policy — 0.6.0 (tenant identity).
  • Cedar / CaMeL — later tiers (primary note §4).
  • TUI backlog (#25/#26/#30/#31/#32) — 0.5.14.
  • JSONL-half erasure (BL-2) — a 0.6.x backlog decision, though the provenance work here is adjacent.

Standing constraints

Unchanged from 0.5.11, plus:

  • The PII reverse map is a secret — never in any sink, log, or serialized record; enforced by the A-1 canary sweep at CI.
  • Rehydration is an egress capability — default-deny; the gate fails closed-on-egress (an unavailable gate keeps the placeholder, never releases).
  • Never enforce on faith — the egress gate ships in observe; enforcement is DECIDE-gated on measured canary + telemetry evidence, matching the project's observe→shadow→enforce posture.
  • config.local.yaml and dev/competitor-tracker.md are owner-owned local artifacts — never commit them.