0.5.14-todo.md

Airlock 0.5.14 — prioritized delivery plan

Release workspace: Slices 0–6 and delivery allocations contain the durable environment, dependency, documentation, requirements, architecture, verification, and HITL review records. Feature implementation begins only after the Slice 6 decision.

This is the release order. A later slice may be designed in parallel, but it does not enter release verification before all acceptance criteria in the prior slice are met. This keeps FathomDB's benchmark dependency ahead of new provider choice, TUI improvements, and the optional operational backend.

Detailed provider design: OpenRouter and DeepSeek integration design defines the provider requirements, architecture findings, and acceptance criteria used by slice 2.

Delivery group A — FathomDB benchmark readiness

Goal: let FathomDB run its planned LLM-backed benchmarks through Airlock without widening the default public model surface, retaining benchmark request/response content in logs, or falsely advertising an embedding endpoint that Airlock does not support.

Work

• gpt-4o-mini chat alias. Add an explicit, reviewed OpenAI-backed gpt-4o-mini model-list entry (and the repository configuration template where applicable). Verify ordinary and streaming chat completions traverse Airlock authentication, guards, alias policy, observability, and served-model attribution. Do not add automatic fallbacks or enable the alias merely because an OpenAI key exists.
• text-embedding-3-small embedding capability and alias. Add the OpenAI-backed embedding alias and make /v1/embeddings a real Airlock capability rather than a configuration-only promise. Preserve ordinary batch inputs and supported dimensions and encoding_format request options; apply the same authentication, policy, observability, error, and redaction boundaries as other inference requests. Extend the capability/model contract and add no-live-credit regression and smoke coverage. Do not advertise embeddings for an alias that has not been configured and tested for it.
• Benchmark-safe log-redaction evidence. Supply a documented benchmark environment profile with AIRLOCK_LOG_REDACT_FIELDS=messages,response and a sentinel-based regression/procedure proving enterprise JSONL stores [REDACTED], not request or response content. Keep SQL logging disabled in that profile because its projection is not content-redacted, and keep AIRLOCK_FATHOM_STORE_MESSAGES and AIRLOCK_FATHOM_STORE_RESPONSE_TEXT disabled. This is separate from the outbound PII guard.

Done when

• FathomDB can use configured gpt-4o-mini chat and text-embedding-3-small embeddings through Airlock's documented interfaces, including a successful manually funded smoke with non-sensitive inputs.
• The model/capability response describes embeddings accurately, and attempts to use an unconfigured embedding model or unsupported option fail clearly.
• Benchmark logging has machine-verifiable redaction evidence and no enabled log or Fathom content-retention path can retain benchmark request/response text.

Delivery group B — provider foundation, OpenRouter, and DeepSeek

Order within the slice: build and test the shared provider boundary first; then integrate OpenRouter; then integrate DeepSeek. The common work is not optional: provider discovery, safe attribution, and sanitized failures must be in place before a provider is presented as supported.

Shared provider foundation

• Preserve explicit model-list authorization: an environment key alone exposes no model, discovery is informational only, and no provider may silently substitute a requested model.
• Add safe configured-API-base catalog discovery, including malformed-base, conflict, no-redirect, and no-secret-in-error behavior.
• Extend served-provider attribution (openrouter is a gateway; deepseek is native) and add a shared bounded provider-error sanitizer before errors enter request events, monitor state, tracing, or enterprise logs.
• Characterize LiteLLM-shaped 401, 402, 429, 500, and 503 outcomes at the Airlock boundary; retain ordinary rate-limit/circuit-breaker behavior without adding provider-specific retry loops or persisting upstream Retry-After.

OpenRouter integration and verification

Why this is scheduled: the pinned LiteLLM 1.94.1 already has a native OpenRouter provider adapter and model registry. This slice is therefore Airlock integration and verification work, not a new provider adapter, transport implementation, or LiteLLM upgrade. It makes the existing capability a first-class, documented deployment option without hiding where routing, fallback, privacy, and billing decisions occur.

Work

• Add a supported configuration recipe that uses LiteLLM's existing openrouter/<vendor>/<model> naming and an environment-provided OPENROUTER_API_KEY with explicit api_base: https://openrouter.ai/api/v1. Document an override only as an explicit operator choice. No key may appear in tracked configuration, logs, TUI state, diagnostics, or test fixtures.
• Extend Airlock's optional, informational startup catalog to discover OpenRouter's /models response and normalize it to the same canonical openrouter/<vendor>/<model> names LiteLLM accepts. Do not duplicate LiteLLM's model registry, use discovery as an authorization source, or silently substitute a different requested model.
• Preserve Airlock's own policy and transparency behavior when OpenRouter performs upstream-provider selection or fallback. Distinguish the requested model from the immediately served openrouter gateway, but do not display OpenRouter upstream-provider/fallback metadata by default or claim that Airlock controls it.
• Verify that LiteLLM's OpenRouter responses retain the signals Airlock needs: 429/503, streaming, and safe served-model/provider metadata. Capture a LiteLLM-exposed Retry-After remains only on the exception response; 0.5.14 does not persist or propagate it to clients. Add a bounded provider error-sanitization seam before OpenRouter streaming-error metadata or user_id can reach JSONL or application logs. Do not reimplement LiteLLM retry, streaming, or response parsing.
• Document the privacy and routing controls an operator must choose deliberately: provider allow/order, fallback behavior, data-collection policy, and zero-data-retention restrictions. Airlock forwards only supported controls and does not claim to enforce an upstream provider's independent retention commitment.
• Add isolated, no-live-credit regression tests for configuration, catalog normalization, safe metadata handling, and Airlock's handling of LiteLLM 429/503 results and Retry-After exception-response boundary. A manually funded smoke test proves the upstream integration; it is not a prerequisite for native LiteLLM support, which is already established by the locked dependency.

Done when

• An operator can configure OpenRouter with a runtime secret and LiteLLM's canonical model name; Airlock's optional catalog displays matching names and the normal policy, observability, and failure-handling paths retain their behavior.
• Regression coverage verifies the supported configuration and Airlock boundary; a documented funded smoke confirms a successful request and streaming without exposing credentials or prompt/response content in artifacts.
• Documentation states the boundary honestly: Airlock governs the request at its proxy, while OpenRouter may select or fall back among its upstream model providers according to the explicitly configured routing/privacy policy.

Provider references: OpenRouter quickstart, provider routing, and error handling.

DeepSeek integration and verification

Why this is scheduled: the pinned LiteLLM 1.94.1 already has a native DeepSeek provider adapter and model registry. This slice is Airlock integration and verification work, not a new provider adapter, custom OpenAI-compatible transport, or LiteLLM upgrade. It documents and validates the deployment path without pinning aliases, leaking an API key or customer identity, or mistaking provider overload for an Airlock policy failure.

Work

• Add a supported configuration recipe using LiteLLM's existing deepseek/<model> naming, DEEPSEEK_API_KEY, and explicit stable api_base: https://api.deepseek.com. LiteLLM 1.94.1 otherwise defaults to a beta endpoint. A key must never appear in tracked configuration, TUI state, diagnostics, or test fixtures; provider error fields must be sanitized before JSONL or application logs.
• Extend Airlock's optional, informational startup catalog to discover DeepSeek's /models response and normalize it to the canonical deepseek/<model> names LiteLLM accepts. Do not make a temporary alias or beta endpoint the default, duplicate LiteLLM's registry, or use discovery as an authorization source.
• Verify standard and streaming requests and OpenAI function tools through the normal Airlock policy, observability, and response-handling paths. Reject a non-function tool before LiteLLM can silently drop it; do not claim JSON mode unless a configured-model regression proves it. Verify that the installed LiteLLM version handles DeepSeek keep-alives; add an Airlock parser only if a focused failing regression proves it is needed.
• Verify the classification presented to Airlock for 401 authentication, 402 balance, 429 concurrency, and transient 500/503 outcomes. Change Airlock handling only for a demonstrated boundary gap; do not persist or propagate an upstream Retry-After, and do not retry an Airlock policy denial.
• Do not forward Airlock's client identity as DeepSeek's optional user_id parameter until a separate privacy review defines a non-identifying, operator-controlled mapping. User IDs must never contain customer or prompt data.
• Add isolated, no-funded-account regression tests for configuration, catalog normalization, the LiteLLM-to-Airlock response boundary, and error classification. A manually funded smoke test confirms upstream operation; it is not a prerequisite for native LiteLLM support, which is already established by the locked dependency.

Done when

• An operator can configure DeepSeek with a runtime secret and LiteLLM's canonical model name; Airlock's optional catalog displays matching names and standard and streaming requests retain normal Airlock behavior.
• Tests prove the LiteLLM-to-Airlock boundary for authentication, balance, rate-limit, and overload results. Health and retry signals remain bounded and never contain credentials, prompts, or a DeepSeek user_id.
• Documentation names the supported stable endpoint and request features, labels beta-only features as opt-in, and states that provider model availability, pricing, and concurrency are controlled by DeepSeek.

Provider references: DeepSeek models, rate limits and user_id, error codes, and tool calls.

Delivery group C — TUI reliability and operator backlog

Order within the slice: first make ordinary TUI tests fast and deterministic; then deliver the bounded operator surfaces below. Each surface must have its source seam before the display work starts, remain off the inference hot path, and degrade visibly when its data is unavailable.

TUI test-harness lifecycle optimization

Why this is scheduled: the 0.5.12 release verification ran uv run pytest --tb=short -q -m 'not live' --durations=40. The suite reached 94% quickly, but the final TUI-heavy segment dominated wall time: every entry in the slowest 40 was a TUI test, at 14.77–27.62 seconds each. The full local suite took 19 minutes.

Each AirlockApp.run_test() constructs all panes and their lifecycle work, not just the view asserted by the test. That includes overview external-probe and state-refresh workers, config/MCP refresh, the JSONL tailer, MCP health loop, and Textual worker cancellation during teardown. This is useful integration coverage in a few tests, but unnecessarily expensive for simple composition, navigation, and rendering assertions.

Work

• Establish an explicit test-harness mode or fixture that prevents irrelevant background I/O and lifecycle workers while retaining the real widget tree.
• Migrate pure TUI composition/navigation/rendering tests to that harness.
• Keep a small, named integration set exercising real worker dispatch, cancellation, shutdown, and the stale-callback guard; do not turn those behaviours into untested mocks.
• Add duration evidence to the closeout and ensure the full non-live suite remains order-independent.

Done when

• pytest --durations=40 shows a material reduction in the TUI-dominated tail.
• Lifecycle-specific tests still cover call_from_thread, worker cancellation, app teardown, JSONL/MCP shutdown, and the OverviewPane stale-refresh path.
• No production TUI behaviour is gated on the test-harness setting.

Deferred 0.5.13 TUI operator backlog

Scope transfer: 0.5.12 deferred this complete set to 0.5.13. There is no separate 0.5.13 plan: 0.5.14 now owns all five issues and their required proxy plumbing. The work remains off the inference hot path; the TUI reads bounded snapshots, JSONL, or authenticated read endpoints rather than subscribing to request callbacks.

Work

• #25 — Smart-router prompt classification. Surface the per-request simple/moderate/complex classification in Guards, its distribution in the operator view, and the tier-to-model mapping that explains the route chosen.
• #26 — Session affinity pins. Add active client pins (model, age, and TTL remaining) to the Clients detail view and a safe operator action for breaking a pin when debugging unexpected routing.
• #30 — Virtual-key management. Deferred to the 0.5.15 backlog with its draft requirements, acceptance criteria, and architecture constraints. It is not 0.5.14 implementation scope.
• #31 — QoS priority during congestion. Show a client's current priority and contributing signals, plus whether congestion makes priority active. Persist only the bounded last-observed information needed by the read view; do not add a TUI dependency to the request path.
• #32 — OpenTelemetry and metrics exporter health. Instrument the exporters before rendering them: independently show enabled state, safely rendered endpoint, last successful export, rolling counts, failures, and a bounded last-error summary. A configuration-only "enabled" indicator is insufficient.
• Add focused unit and integration coverage for every new snapshot/read seam, stale or unavailable data, and the destructive/key-secret paths. Close each issue with evidence, or leave it open with an explicit missing prerequisite.

Done when

• Issues #25, #26, #31, and #32 are closed with the documented operator surface and evidence, or retain an explicit, evidence-backed blocker. Issue #30 is explicitly deferred to 0.5.15.
• New read paths remain bounded and degrade visibly when their source is unavailable; none run on the inference hot path.
• Key material and exporter credentials are never rendered, retained in a UI snapshot, or written to Airlock telemetry.

Delivery group D — FathomDB as an optional operational backend

Why this is scheduled: 0.5.11/0.5.12 made FathomDB an optional, lazy, single-owner request-analysis store. Slice 110 makes it a selectable operational-read backend through a proxy-owned, loopback-only bridge, while preserving JSONL as the default and fallback.

Work

• Implemented in Slice 110; see its design and status record for the supported readers, bounds, bridge authorization, fallback behavior, and retained erasure-scope limitation.

Done when

• Complete: the status record contains the durable acceptance evidence.