keyprovider: decide direct TPM2 ESYS provider follow-up

[justinjoy]

Summary

Store RBAC permission data in a standalone encrypted file (separate from the DuckDB telemetry store) with a pluggable key-provider abstraction. Only two provider backends are supported:

1. systemd-creds (default, no library linkage — invoked as a subprocess)
2. tpm2-tss with the mbedtls crypto backend (for environments that need direct TPM 2.0 binding without systemd)

No OpenSSL. No other backends (no libsecret, no passphrase prompt, no PKCS#11, no cloud KMS) are in scope for this issue.

Motivation

• RBAC data is small, slow-changing, and not joined with telemetry — no benefit from placing it inside DuckDB.
• DuckDB's native at-rest encryption requires the key to appear as a literal inside an ATTACH SQL statement, which complicates key hygiene and has no dedicated C API.
• An independent encrypted RBAC file allows key material to be unwrapped only at load time, kept in locked memory, and zeroed immediately after use.
• wyrelog's GPL-3.0-or-later + Commercial dual license requires permissive or compatible dependencies; OpenSSL is excluded by project policy.

Scope

File format

• Container: libsodium crypto_secretstream_xchacha20poly1305 (AEAD, chunked)
• Atomic writes: temp file + fsync + rename
• File mode: 0600, dedicated UID/GID
• Header carries format version and provider identifier (systemd-creds | tpm2) so a file can be opened by the correct unwrap path without configuration lookup

KeyProvider interface (C)

typedef struct _WyrelogKeyProvider WyrelogKeyProvider;

gboolean wyrelog_key_provider_unwrap (WyrelogKeyProvider *provider,
                                      guint8             *dek_out,     /* 32 bytes */
                                      GError            **error);

gboolean wyrelog_key_provider_wrap   (WyrelogKeyProvider *provider,
                                      const guint8       *dek_in,      /* 32 bytes */
                                      GError            **error);

• DEK buffers allocated via sodium_malloc + sodium_mlock
• sodium_memzero on free, no logging of key material or SQL/command lines containing keys

Backend: systemd-creds

• No library linkage; wyrelog spawns systemd-creds encrypt/decrypt with the DEK on stdin/stdout
• On TPM-equipped hosts systemd transparently seals to TPM2; otherwise falls back to the host key
• Minimum systemd version documented (249+)

Backend: tpm2-tss + mbedtls

• Build option --with-crypto=mbed on tpm2-tss
• Seal DEK under the owner hierarchy SRK; optional PCR policy (documented but off by default to avoid operational surprises across kernel/firmware updates)
• Unseal path uses the ESYS API; sealed blob stored next to the RBAC file

Meson wiring

• New dependencies: libsodium (unconditional), tpm2-tss ESYS (optional, feature-flagged -Dtpm2=enabled)
• Build matrix adds one job per provider
• THIRD_PARTY_NOTICES.md seeded with libsodium (ISC), mbedtls (Apache-2.0), and tpm2-tss (BSD-2-Clause) entries

Out of scope

• Passphrase / libsecret / PKCS#11 / cloud KMS providers
• OpenSSL-based TPM stacks (tpm2-openssl, tpm2-tss-engine)
• Key rotation UX beyond a single wyrelog rbac rotate-key command that re-encrypts in place
• Placing RBAC inside DuckDB

Acceptance criteria

• WyrelogKeyProvider interface and two concrete backends land behind build flags
• RBAC file reader/writer uses libsodium secretstream with atomic rename
• Unit tests cover round-trip for each provider; tpm2 test gated on a CI job with a software TPM (swtpm)
• DEK lifetime is bounded — verified by test that memory-dumps the RBAC manager after close() and asserts no 32-byte key remains
• THIRD_PARTY_NOTICES.md present and CI verifies it is updated when dependencies change
• Documentation: operator guide for choosing a provider and rotating keys

References

• DuckDB at-rest encryption caveats: https://duckdb.org/2025/11/19/encryption-in-duckdb
• systemd-creds: https://www.freedesktop.org/software/systemd/man/systemd-creds.html
• tpm2-tss mbedtls backend: https://github.com/tpm2-software/tpm2-tss

[justinjoy]

Codebase sync — 2026-05-12

Current main has implemented the encrypted policy-store and production KeyProvider path, but this issue still contains one unresolved design item, so I am leaving it open rather than closing it as completed.

Implemented now:

• Encrypted policy-store open path for production mode.
• Stable encrypted store header/provider identity and authenticated metadata handling.
• Wrong key/provider, corrupt header, unsupported version, invalid provider id, truncated ciphertext, and invalid metadata fail closed.
• Atomic private persistence path with temp file, 0600 mode, fsync/rename behavior where supported.
• Offline encrypted-store key rotation through wyctl key rotate.
• Production KeyProvider specs: systemd-creds:NAME and file:PATH/bare path for manual bootstrap checks.
• Packaged systemd units pass production policy roots through LoadCredential= and default to systemd-creds: specs.
• wyctl key status exposes operator-facing provider readiness.

Evidence in-tree:

• wyrelog/policy/store.c
• wyrelog/policy/store-private.h
• wyrelog/wyl-keyprovider-file.c
• wyrelog/wyl-keyprovider-file-private.h
• wyrelog/daemon/options.c
• wyrelog/wyctl/wyctl.c
• packaging/systemd/wyrelog-system.service
• packaging/systemd/wyrelog-service.service
• tests/test-policy-store.c
• tests/test-keyprovider-file.c
• tests/check-wyrelogd-production-gates.sh
• tests/check-packaged-install-readiness.sh
• docs/operator-runbook.md

Still not implemented as written in the issue:

• A direct tpm2-tss KeyProvider backend is not present. The current production path relies on systemd credential delivery; TPM binding can be provided by the host/systemd layer when configured that way, but there is no Wyrelog-native ESYS seal/unseal provider yet.
• The issue also describes some earlier API naming/details that no longer match the current internal API names.

Recommended current tracking:

• Keep this issue open only for the direct TPM-provider decision, or retitle/split it into a focused follow-up such as “keyprovider: add direct TPM2 ESYS provider”.
• If the current v0 decision is that systemd-creds is the only production provider and file: is bootstrap/manual-check only, then this issue can be closed as superseded after that decision is recorded.

Validation observed during the latest local sync:

• meson test -C builddir --suite wyrelog --print-errorlogs → 59 OK / 0 FAIL / 1 SKIP.

[justinjoy]

Closing — decision recorded; residual defect spun out as successor

Architect + critic re-review on 2026-05-19. Both converged on SUCCESSOR.

Decision (recorded for the record)

v0 production KeyProvider posture:

• systemd-creds:NAME is the only production provider. TPM binding is delegated to the host's systemd-creds stack via LoadCredential= in the packaged units (packaging/systemd/wyrelog-system.service, wyrelog-service.service).
• file:PATH is bootstrap/manual-check only.
• No native ESYS provider in v0. Duplicating the TPM seal path inside wyrelog would mean owning SRK/PCR policy ergonomics that the host distribution already owns, a swtpm-in-CI burden, and additional third-party-notice surface on a GPL-3.0+/commercial dual-licensed binary — for no concrete consumer need.
• Decision is reversible: if a deployment ever needs non-systemd TPM binding, a fresh, scoped issue is cheap. Do not pre-open speculative tracking.

Implemented work (per the 2026-05-12 sync, still accurate)

The encrypted policy-store, atomic persistence, header/provider identity, fail-closed error paths, wyctl key rotate, and operator UX (wyctl key status) are all landed.

Residual defect — spun out as a focused successor

The repo currently ships build machinery that advertises a TPM-backed provider that doesn't exist:

• meson.build:19 declares tss2_esys_dep = dependency('tss2-esys', required : get_option('enable_tpm')) — repo-wide grep finds zero consumers of tss2_esys_dep or Esys_* symbols.
• meson.options enable_tpm description claims wyrelog "builds the hardware-backed key provider if available" — false.
• meson.options require_tpm (default true) gates a fail-closed production check pointing at a non-existent provider.

This is a documentation/build lie that will mislead packagers and security auditors. Tracked separately as the successor issue (linked below).

Closing as superseded by the implemented work; the dead-option cleanup is the only actionable residue.

[justinjoy]

Successor: #330 — keyprovider: remove unused enable_tpm/require_tpm/tss2-esys build machinery

[justinjoy]

Correction — reopening; prior close mischaracterised intent

The previous close comment framed the direct TPM2 ESYS KeyProvider as "deferred indefinitely / reopen when a concrete deployment requires it." That mischaracterised the project intent.

Correct status: the direct ESYS KeyProvider is on hold (planned, not now), not rejected and not contingent on a future external request. It is queued work; the timing is what's open, not the yes/no.

Implications of this correction:

• The enable_tpm / require_tpm options in meson.options and the tss2_esys_dep declaration at meson.build:19 are placeholder build machinery for the planned ESYS provider — they are not dead code, and they should not be removed. The previously-filed successor proposing their deletion will be retracted.
• This issue stays open as the tracking record for the planned ESYS provider work. The implemented portions (encrypted store, file: / systemd-creds: specs, LoadCredential=, wyctl key rotate/status) remain landed as listed in the 2026-05-12 sync; the residual scope is the ESYS implementation itself.

Apologies for the noise on the prior close.