Merge pull request #108 from runcycles/fix/audit-log-retention-ttl-v0.1.25.15

fix(audit): retention TTL on runtime-written audit rows (v0.1.25.15)

Author: amavashev

AUDIT.md

@@ -1,6 +1,7 @@
 # Cycles Protocol v0.1.25 — Server Implementation Audit
 
-**Date:** 2026-04-18 (v0.1.25.14 — trace_id (W3C Trace Context) cross-surface correlation per cycles-protocol revision 2026-04-18; new `TraceContextFilter` extracts `traceparent` or `X-Cycles-Trace-Id` from inbound requests or generates a fresh 128-bit id, echoes `X-Cycles-Trace-Id` on every response, populates `trace_id` on `ErrorResponse` / `Event` / `WebhookDelivery` / `AuditLogEntry`),
+**Date:** 2026-04-18 (v0.1.25.15 — runtime audit-log retention TTL fix; `AuditRepository` now writes `audit:log:{id}` keys with `EX ttl` via the same Lua shape admin uses, configurable via `audit.retention.days` (default 400d), daily `@Scheduled` sweep prunes stale ZSET index pointers; closes a gap where runtime-written rows persisted indefinitely and did not participate in admin's authenticated-tier retention),
+2026-04-18 (v0.1.25.14 — trace_id (W3C Trace Context) cross-surface correlation per cycles-protocol revision 2026-04-18; new `TraceContextFilter` extracts `traceparent` or `X-Cycles-Trace-Id` from inbound requests or generates a fresh 128-bit id, echoes `X-Cycles-Trace-Id` on every response, populates `trace_id` on `ErrorResponse` / `Event` / `WebhookDelivery` / `AuditLogEntry`),
 2026-04-16 (v0.1.25.13 — hydration cap + enum wire annotations on the sorted `GET /v1/reservations` path; `SORTED_HYDRATE_CAP=2000` guard on the in-memory sort hydration with WARN-on-cap, matches admin plane's v0.1.25.24 pattern; `@JsonValue`/`@JsonCreator fromWire` on `ReservationSortBy` + `SortDirection` to mirror admin's `SortSpec`/`SortDirection` contract),
 2026-04-16 (v0.1.25.12 — `sort_by` + `sort_dir` on `GET /v1/reservations` per cycles-protocol spec revision 2026-04-16; 7-value sort enum, opaque cursor binds `(sort_by, sort_dir, filters)` tuple, legacy SCAN-cursor path preserved when both params omitted),
 2026-04-14 (automated performance regression detection — nightly trend + release gate, no version bump),
@@ -20,6 +21,31 @@
 
 ---
 
+### 2026-04-18 — v0.1.25.15: audit-log retention TTL (runtime-side fix)
+
+Closes a gap surfaced by the post-v0.1.25.14 alignment audit: runtime's `AuditRepository.log()` was writing `audit:log:{id}` string keys with no `EX`, so runtime-written audit rows persisted indefinitely until Redis memory-eviction kicked in. This broke the 400-day retention story the admin plane tells operators — admin's `AuditRepository` already applies tiered TTL (400d authenticated / 30d unauthenticated) via a conditional Lua `SET … EX ttl`, but runtime-written rows were silently non-compliant with that contract. The audit dashboard reads from the shared index, so stale admin ZSETs would also accumulate pointers to long-expired runtime rows without a compensating sweep.
+
+**Root cause:** the original v0.1.25.8 introduction of runtime-side audit writes copied admin's Lua shape from *before* admin added per-entry TTL in its v0.1.25.20. Admin's TTL work never back-propagated to runtime — not a regression, just an unnoticed drift.
+
+**Scope decision — runtime-side fix over admin-side reconciliation:** the writer should own retention, not a downstream sweeper. Admin-side reconciliation would have required a reaper polling for TTL-less keys — heavier, fights the symptom, couples admin's cleanup cadence to runtime's write rate. Runtime-side adds ~100 LOC (Lua arg + config + sweeper mirror) with zero API surface change.
+
+**Scope decision — single tier instead of admin's two tiers:** runtime only writes real-tenant audit rows. The `__admin__` (platform-plane) and `__unauth__` (pre-auth failure) sentinels are admin-plane concerns — runtime authenticates every request before the audit write, and runtime-plane operations like reservation release are always tenant-attributed. So one `audit.retention.days` knob (default 400 to match admin's authenticated tier) is sufficient. If a future runtime endpoint ever needs the 30-day short-tier behavior, this config can be extended without wire change.
+
+**Lua change:** `LOG_AUDIT_LUA` now reads `ARGV[4]` as an optional TTL in seconds; conditional branch matches admin's script byte-for-byte (minus the sentinel logic). Atomic guarantee preserved — SET + 2×ZADD still run in one call, so the TTL addition cannot introduce orphaned index pointers on a mid-write crash.
+
+**Sweeper:** mirrors admin's `sweepStaleIndexEntries()` — daily @Scheduled cron (default `0 0 3 * * *`), `ZREMRANGEBYSCORE` on `audit:logs:_all` plus SCAN over per-tenant indexes. Runtime deploys a sweeper of its own (rather than depending on admin's sweep hitting the shared Redis) so a runtime-only topology stays self-healing. Two sweepers scanning the same index is idempotent — `ZREMRANGEBYSCORE` on already-swept ranges is a no-op.
+
+**Config:**
+
+- `audit.retention.days` (env `AUDIT_RETENTION_DAYS`) — default `400`. `0` = indefinite retention; the sweeper is skipped explicitly in that mode (matching admin's `authenticatedRetentionDays=0` behavior).
+- `audit.sweep.cron` (env `AUDIT_SWEEP_CRON`) — default `0 0 3 * * *` (03:00 server time).
+
+**Backward compatibility:** new writes get TTL; old runtime-written keys stay un-TTL'd until Redis memory pressure evicts them. No API change, no wire change, no admin-side reconciliation needed — admin's reader doesn't care whether the target key has a TTL, it only cares that a value is present or not (and already null-body-tolerant for TTL-expired pointers).
+
+**Tests:** 7-case `AuditRepositoryTest` covering: (1) TTL passed as ARGV[4] = 400×86400, (2) `retentionDays=0` passes `"0"`, (3) `logId` + timestamp set on the entry, (4) Redis failure is non-fatal, (5) sweeper removes global and per-tenant pointers, (6) sweeper is a no-op when retention is 0, (7) sweep Redis failure is non-fatal. All green; full data module still 365 / 365; full api module still 152 / 152.
+
+---
+
 ### 2026-04-18 — v0.1.25.14: trace_id cross-surface correlation (W3C Trace Context)
 
 Implements the `CORRELATION AND TRACING` normative section added to `cycles-protocol-v0.yaml` in spec revision 2026-04-18 (commit `8d65959`). Introduces a third correlation identifier — `trace_id` — that is W3C Trace Context-compatible (OpenTelemetry-native) and links every HTTP request to its `ErrorResponse`, audit-log entry, emitted events, and outbound webhook deliveries under one logical-operation grain.

CHANGELOG.md

@@ -14,6 +14,39 @@ changes to request/response bodies or Lua-script semantics would require a
 minor bump. "Internal signature changes" (e.g. Java method parameters) are
 called out but are not breaking to API clients.
 
+## [0.1.25.15] — 2026-04-18
+
+### Fixed
+
+- Runtime-written audit-log entries now respect a configurable retention
+  TTL (default 400 days). Previously, `AuditRepository.log()` wrote
+  `audit:log:{id}` keys with no `EXPIRE`, so runtime-written rows
+  persisted indefinitely until Redis eviction — silently failing to
+  participate in the 400-day retention tier the admin plane applies to
+  authenticated audit rows. Matches the authenticated-tier default on
+  `cycles-server-admin`'s `AuditRepository` (admin's
+  `audit.retention.authenticated.days=400`). Runtime never writes the
+  admin-plane `__admin__` / `__unauth__` sentinels, so a single tier
+  is sufficient.
+
+### Added
+
+- `audit.retention.days` config (default `400`, env `AUDIT_RETENTION_DAYS`).
+  Set to `0` for indefinite retention (legal hold, HIPAA-adjacent
+  deployments, or environments that offload audit to an archive store).
+- `audit.sweep.cron` config (default `0 0 3 * * *`, env `AUDIT_SWEEP_CRON`).
+  Daily `@Scheduled` sweep prunes stale `audit:logs:{tenantId}` and
+  `audit:logs:_all` ZSET pointers whose target `audit:log:{id}` key has
+  TTL-expired. Self-contained — does not depend on admin's sweep running
+  against the same Redis. Safe to run in parallel with admin's sweep
+  (idempotent `ZREMRANGEBYSCORE`).
+
+### Internal
+
+- `AuditRepository.LOG_AUDIT_LUA` now reads ARGV[4] as an optional TTL
+  in seconds (`0` or negative = no `EX`). Same shape as admin's script,
+  minus the sentinel branching.
+
 ## [0.1.25.14] — 2026-04-18
 
 ### Added

OPERATIONS.md

@@ -414,6 +414,8 @@ don't fit.
 | `cycles.expiry.initial-delay-ms` | `5000` | Mostly a test knob. Leave. |
 | `cycles.tenant-config.cache-ttl-ms` | `60000` | Lower if admin tenant config changes need to take effect faster than 60s. |
 | `admin.api-key` | (empty) | Set to a fixed-length secret to enable the admin-on-behalf-of endpoint (v0.1.25.8+). Leave empty to disable. |
+| `audit.retention.days` | `400` | Retention for runtime-written audit rows (v0.1.25.15+). Default matches admin's `audit.retention.authenticated.days` — SOC2 Type II 12-month lookback + 1-month auditor-lag buffer. Set `0` for indefinite retention (legal hold, archive-store deployments). |
+| `audit.sweep.cron` | `0 0 3 * * *` | Daily cron for pruning stale ZSET index pointers (v0.1.25.15+). Lower cadence if audit write volume is very high; leave as-is otherwise. Skipped when `audit.retention.days=0`. |
 | `management.endpoints.web.exposure.include` | `health,info,prometheus` | Add more actuator endpoints if you need them, but `prometheus` is the one ops cares about. |
 
 ## Reservation list sorting (v0.1.25.12+)

cycles-protocol-service/cycles-protocol-service-api/src/main/resources/application.properties

@@ -58,3 +58,9 @@ admin.api-key=${ADMIN_API_KEY:}
 # Event/delivery retention TTL
 events.retention.event-ttl-days=${EVENT_TTL_DAYS:90}
 events.retention.delivery-ttl-days=${DELIVERY_TTL_DAYS:14}
+
+# Audit log retention (v0.1.25.15+). Matches admin's authenticated tier
+# default. Set to 0 for indefinite retention. audit.sweep.cron prunes
+# stale ZSET index pointers whose target key has TTL-expired.
+audit.retention.days=${AUDIT_RETENTION_DAYS:400}
+audit.sweep.cron=${AUDIT_SWEEP_CRON:0 0 3 * * *}

cycles-protocol-service/cycles-protocol-service-data/src/main/java/io/runcycles/protocol/data/repository/AuditRepository.java

@@ -5,9 +5,13 @@
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.scheduling.annotation.Scheduled;
 import org.springframework.stereotype.Repository;
 import redis.clients.jedis.Jedis;
 import redis.clients.jedis.JedisPool;
+import redis.clients.jedis.params.ScanParams;
+import redis.clients.jedis.resps.ScanResult;
 
 import java.time.Instant;
 import java.util.List;
@@ -25,10 +29,19 @@
  * 2026-04-13's NORMATIVE requirement that admin-driven releases
  * record an audit-log entry with {@code actor_type=admin_on_behalf_of}.
  *
+ * <p>v0.1.25.15: audit entries now respect a configurable TTL
+ * (default 400 days — SOC2 Type II 12-month lookback + buffer).
+ * Matches the authenticated-tier retention admin's
+ * {@code io.runcycles.admin.data.repository.AuditRepository} applies to
+ * real-tenant rows. Runtime never writes the admin-plane sentinels
+ * ({@code __admin__}, {@code __unauth__}) so a single tier suffices.
+ * A daily sweep prunes stale index pointers whose target string key
+ * has TTL-expired.
+ *
  * <p>Key layout (MUST match {@code
  * io.runcycles.admin.data.repository.AuditRepository}):
  * <ul>
- *   <li>{@code audit:log:<log_id>} — JSON-serialized entry</li>
+ *   <li>{@code audit:log:<log_id>} — JSON-serialized entry, EX = ttl</li>
  *   <li>{@code audit:logs:<tenant_id>} — per-tenant ZSET index,
  *       score = timestamp millis</li>
  *   <li>{@code audit:logs:_all} — global ZSET index</li>
@@ -46,10 +59,40 @@
 public class AuditRepository {
     private static final Logger LOG = LoggerFactory.getLogger(AuditRepository.class);
 
-    // Same Lua script as cycles-server-admin's AuditRepository.
-    // Identical key layout and atomicity guarantees.
+    /**
+     * Retention for runtime-written audit entries. Default 400 days =
+     * SOC2 Type II 12-month lookback + 1-month buffer for post-period
+     * auditor lag. Set to {@code 0} for indefinite retention (legal
+     * hold, HIPAA-adjacent deployments, or environments that offload
+     * to an archive store).
+     *
+     * <p>Matches {@code audit.retention.authenticated.days} on the
+     * admin plane — runtime only writes real-tenant rows, so a single
+     * tier is sufficient.
+     *
+     * @since 0.1.25.15
+     */
+    @Value("${audit.retention.days:400}")
+    private int retentionDays;
+
+    /**
+     * Lua script for atomic audit-log creation with optional TTL via
+     * ARGV[4] (seconds; 0 or negative = no expiry). Identical shape to
+     * admin's script — keeps the wire-compatible storage layout the
+     * admin dashboard reads against.
+     * <pre>
+     *   SET audit:log:&#123;logId&#125; &lt;json&gt; [EX ttlSeconds]
+     *   ZADD audit:logs:&#123;tenantId&#125; &lt;score&gt; &lt;logId&gt;
+     *   ZADD audit:logs:_all         &lt;score&gt; &lt;logId&gt;
+     * </pre>
+     */
     private static final String LOG_AUDIT_LUA =
-        "redis.call('SET', KEYS[1], ARGV[1])\n" +
+        "local ttl = tonumber(ARGV[4])\n" +
+        "if ttl and ttl > 0 then\n" +
+        "    redis.call('SET', KEYS[1], ARGV[1], 'EX', ttl)\n" +
+        "else\n" +
+        "    redis.call('SET', KEYS[1], ARGV[1])\n" +
+        "end\n" +
         "redis.call('ZADD', KEYS[2], ARGV[2], ARGV[3])\n" +
         "redis.call('ZADD', KEYS[3], ARGV[2], ARGV[3])\n" +
         "return 1\n";
@@ -64,15 +107,72 @@ public void log(AuditLogEntry entry) {
             entry.setTimestamp(Instant.now());
             String json = objectMapper.writeValueAsString(entry);
             String score = String.valueOf(entry.getTimestamp().toEpochMilli());
+            long ttlSeconds = retentionDays > 0 ? (long) retentionDays * 86400L : 0L;
             jedis.eval(LOG_AUDIT_LUA,
                 List.of("audit:log:" + logId,
                         "audit:logs:" + entry.getTenantId(),
                         "audit:logs:_all"),
-                List.of(json, score, logId));
+                List.of(json, score, logId, String.valueOf(ttlSeconds)));
         } catch (Exception e) {
             // Audit log failure must NOT break the business operation.
             LOG.error("Failed to write audit log (non-fatal): operation={} resource_id={}",
                 entry.getOperation(), entry.getResourceId(), e);
         }
     }
+
+    /**
+     * Daily sweep of the audit sorted-set indexes to remove pointers whose
+     * target {@code audit:log:{id}} key has already expired. Without this
+     * sweep the {@code audit:logs:_all} and per-tenant sorted sets would
+     * grow unbounded even though the underlying log records are gone —
+     * stale pointers still cost memory and lengthen any read-side scan.
+     *
+     * <p>Mirrors admin's {@code sweepStaleIndexEntries()} so runtime's
+     * cleanup is self-contained (does not depend on admin being deployed
+     * against the same Redis). Both sweeps scanning the shared index
+     * is idempotent and safe — {@code ZREMRANGEBYSCORE} is a no-op on
+     * already-swept ranges.
+     *
+     * <p>Runs at 03:00 server time by default (configurable via
+     * {@code audit.sweep.cron}). Deployments with
+     * {@code audit.retention.days=0} skip the sweep.
+     *
+     * <p>Best-effort: any exception is logged at ERROR but never
+     * propagates. Skipped on the current tick if Redis is unavailable —
+     * the next tick retries.
+     *
+     * @since 0.1.25.15
+     */
+    @Scheduled(cron = "${audit.sweep.cron:0 0 3 * * *}")
+    public void sweepStaleIndexEntries() {
+        if (retentionDays <= 0) {
+            LOG.debug("Audit index sweep skipped — retention is indefinite");
+            return;
+        }
+        try (Jedis jedis = jedisPool.getResource()) {
+            long cutoffMillis = Instant.now().toEpochMilli()
+                    - ((long) retentionDays * 86400L * 1000L);
+            long removedGlobal = jedis.zremrangeByScore("audit:logs:_all",
+                    Double.NEGATIVE_INFINITY, cutoffMillis);
+            long removedTenants = 0;
+            ScanParams params = new ScanParams().match("audit:logs:*").count(100);
+            String cursor = ScanParams.SCAN_POINTER_START;
+            do {
+                ScanResult<String> scan = jedis.scan(cursor, params);
+                for (String indexKey : scan.getResult()) {
+                    if ("audit:logs:_all".equals(indexKey)) {
+                        continue;
+                    }
+                    removedTenants += jedis.zremrangeByScore(indexKey,
+                            Double.NEGATIVE_INFINITY, cutoffMillis);
+                }
+                cursor = scan.getCursor();
+            } while (!ScanParams.SCAN_POINTER_START.equals(cursor));
+            LOG.info("Audit index sweep completed — removed {} global + {} per-tenant stale pointers "
+                            + "older than {} ms",
+                    removedGlobal, removedTenants, cutoffMillis);
+        } catch (Exception e) {
+            LOG.error("Audit index sweep failed (non-fatal — next tick will retry)", e);
+        }
+    }
 }