perf(bench): baseline sorted reservation scaling (v0.1.25.53)

Author: amavashev

AUDIT.md

@@ -5,6 +5,44 @@
 
 ---
 
+### 2026-07-14 — v0.1.25.53: sorted-list scaling baseline
+
+This benchmark-only release is Phase A of #240. The sorted reservation path
+still performs its correctness-preserving global SCAN, tenant/filter hydration,
+in-memory sort, and cursor slice; no production repository or Lua behavior is
+changed.
+
+`CyclesProtocolReadBenchmarkTest` now seeds 1,000- and 10,000-row populations
+directly through pipelined Redis hashes, with half the rows belonging to the
+authenticated tenant and half to an unrelated tenant. Each measured request is
+the default sorted query (`created_at_ms desc`, `limit=20`). Three same-host
+trials produced median p50/p95/p99 of 22.5/40.9/48.3 ms at 1,000 total rows and
+164.9/210.2/232.0 ms at 10,000. The 10k path hydrates 5,000 matching rows to
+return 20, confirming the next read optimization target without relying on the
+old truncating 2,000-row cap.
+
+The parser, median aggregator, regression checker, and benchmark-data format
+now carry `list_sorted_1k_p50_ms` and `list_sorted_10k_p50_ms` as gating latency
+metrics. Shipping this measurement before the runtime index gives Phase B a
+real preceding-release baseline. The design note was rewritten around one
+default-sort ZSET with completeness count validation, explicit post-rollout
+readiness, restartable backfill, stale-member cleanup, bounded hydration, and a
+full-SCAN fallback; it explicitly rejects the permanent-marker failure modes
+removed from #235.
+
+Version/revision 0.1.25.52 → 0.1.25.53. Both production compose variants pin
+the matching image. There is no public schema, successful-response, or Redis
+storage change.
+
+**Validation.** `mvn clean verify -Pintegration-tests` completed 1,144 tests
+(31 model, 534 data, 579 API) with zero failures, errors, or skips; the
+authoritative protocol coverage check remained 11/11. JaCoCo line coverage is
+95.12% data and 95.56% API. The exact release benchmark command completed all
+17 cases with zero errors, emitted both new metrics, and parsed all nine
+headline values successfully. Focused three-run medians are recorded in
+`BENCHMARKS.md`. Both production Compose files pass `docker compose config
+--quiet` with their required variables supplied.
+
 ### 2026-07-14 — v0.1.25.52: replay-storage and ledger-helper follow-up
 
 This non-behavioral follow-up closes the two intentionally deferred cleanup

BENCHMARKS.md

@@ -27,8 +27,8 @@ running them would only measure environmental noise. Skipped releases:
   list endpoint. The sorted path is opt-in (clients must pass the
   new params to activate it); legacy list behaviour is byte-for-byte
   unchanged and all existing benchmarks exercise the legacy path.
-  Benchmarks for the sorted path are worth adding once real tenant
-  populations exercise the O(N) full-SCAN; see OPERATIONS.md.
+  Population benchmarks for the sorted path were added after v0.1.25.52 under
+  #240; see the frozen pre-index baseline below.
 - **v0.1.25.13** — hydration cap + enum wire annotations on the sorted
   list path. Write-path unchanged.
 - **v0.1.25.14** — trace_id (W3C Trace Context) correlation. The new
@@ -39,7 +39,34 @@ running them would only measure environmental noise. Skipped releases:
   Benchmarks deliberately skipped — they would only measure
   environmental noise. [benchmark-skip]
 
-Last benchmarked release: **v0.1.25.52**.
+Last benchmarked release: **v0.1.25.53**.
+
+---
+
+## v0.1.25.53 — Sorted-list scaling baseline
+
+**Date:** 2026-07-14
+
+**Tag:** `v0.1.25.53` *(benchmark-only follow-up; tag after merge)*
+
+**Environment:** Windows 11 Pro for Workstations, AMD Ryzen Threadripper 3990X
+64-Core, Java 21, Docker + Redis 7 (Testcontainers)
+
+**Method:** The runtime is unchanged from v0.1.25.52. Each fixture contains
+equal numbers of authenticated-tenant and unrelated-tenant reservation hashes,
+requests `limit=20&sort_by=created_at_ms&sort_dir=desc`, and measures 200
+requests after 50 warmups. Fixture writes are pipelined and complete before
+timing starts. Values are medians of three same-host trials.
+
+| Total rows | Target-tenant rows | p50 | p95 | p99 |
+|---:|---:|---:|---:|---:|
+| 1,000 | 500 | 22.5ms | 40.9ms | 48.3ms |
+| 10,000 | 5,000 | 164.9ms | 210.2ms | 232.0ms |
+
+**Assessment:** The 7.3x p50 increase for a 10x population confirms the
+global-SCAN/hydrate/sort path is the next material read bottleneck. These two
+p50s are parsed into nightly and release benchmark data so #240's planned
+indexed path has a machine-readable preceding-release comparison point.
 
 ---
 

CHANGELOG.md

@@ -14,6 +14,25 @@ changes to request/response bodies or Lua-script semantics would require a
 minor bump. "Internal signature changes" (e.g. Java method parameters) are
 called out but are not breaking to API clients.
 
+## [0.1.25.53] — 2026-07-14
+
+### Performance measurement
+
+- **Sorted reservation listing has population baselines.** The benchmark suite
+  now measures the default `created_at_ms desc` sorted-list query at 1,000 and
+  10,000 total reservation hashes, split evenly between the authenticated and
+  an unrelated tenant. Both p50 values are tracked by nightly and release
+  regression checks, establishing the pre-index comparison point for #240.
+- Benchmark fixture writes are pipelined and complete before timing begins.
+  Expected large-hydration warnings are suppressed only in the benchmark
+  profile so console I/O does not distort the measurement.
+
+### Compatibility
+
+- No production request-path, Redis-data, or wire-format behavior changes.
+  Production and full-stack compose defaults self-pin
+  `ghcr.io/runcycles/cycles-server:0.1.25.53`.
+
 ## [0.1.25.52] — 2026-07-14
 
 ### Performance

benchmarks/README.md

@@ -44,7 +44,7 @@ https://raw.githubusercontent.com/runcycles/cycles-server/benchmark-data/benchma
 
 ## Headline metrics tracked
 
-Seven numbers chosen for signal density, not exhaustiveness. More metrics
+Nine numbers chosen for signal density, not exhaustiveness. More metrics
 means more noise and a higher false-positive rate on the gate.
 
 | Metric | Source test | Why |
@@ -53,14 +53,16 @@ means more noise and a higher false-positive rate on the gate.
 | `commit_p50_ms`, `commit_p99_ms` | `CyclesProtocolBenchmarkTest` | Second write path |
 | `release_p50_ms` | `CyclesProtocolBenchmarkTest` | Cleanup path |
 | `event_p50_ms` | `CyclesProtocolBenchmarkTest` | Direct-debit path |
+| `list_sorted_1k_p50_ms` | `CyclesProtocolReadBenchmarkTest` | Sorted-list scaling at moderate population |
+| `list_sorted_10k_p50_ms` | `CyclesProtocolReadBenchmarkTest` | Sorted-list scaling trigger / indexed-path payoff |
 | `concurrent_throughput_32t` | `CyclesProtocolConcurrentBenchmarkTest` | Scaling signal |
 
 ## Entry format (`history.jsonl`)
 
 Each line is a standalone JSON object:
 
 ```json
-{"timestamp":"2026-04-15T07:00:00Z","commit":"abc1234","tag":null,"reserve_p50_ms":5.3,"reserve_p99_ms":18.2,"commit_p50_ms":4.6,"commit_p99_ms":15.1,"release_p50_ms":4.8,"event_p50_ms":4.3,"concurrent_throughput_32t":2632}
+{"timestamp":"2026-07-15T07:00:00Z","commit":"abc1234","tag":null,"reserve_p50_ms":5.3,"reserve_p99_ms":18.2,"commit_p50_ms":4.6,"commit_p99_ms":15.1,"release_p50_ms":4.8,"event_p50_ms":4.3,"list_sorted_1k_p50_ms":22.5,"list_sorted_10k_p50_ms":164.9,"concurrent_throughput_32t":2632}
 ```
 
 - `timestamp` — UTC, ISO 8601

cycles-protocol-service/cycles-protocol-service-api/src/test/java/io/runcycles/protocol/api/CyclesProtocolReadBenchmarkTest.java

@@ -3,6 +3,8 @@
 import org.junit.jupiter.api.*;
 import org.springframework.http.ResponseEntity;
 import org.springframework.test.context.ActiveProfiles;
+import redis.clients.jedis.Jedis;
+import redis.clients.jedis.Pipeline;
 
 import java.util.*;
 import java.util.stream.LongStream;
@@ -78,6 +80,18 @@ void benchmarkListReservations() {
                 timings[0], timings[timings.length - 1], mean(timings)));
     }
 
+    @Test
+    @DisplayName("GET /v1/reservations sorted at 1k total rows")
+    void benchmarkSortedReservationsAt1k() {
+        benchmarkSortedReservations(1_000, "LIST sorted @1k");
+    }
+
+    @Test
+    @DisplayName("GET /v1/reservations sorted at 10k total rows")
+    void benchmarkSortedReservationsAt10k() {
+        benchmarkSortedReservations(10_000, "LIST sorted @10k");
+    }
+
     @Test
     @DisplayName("GET /v1/balances")
     void benchmarkGetBalances() {
@@ -152,4 +166,50 @@ private Map<String, Object> decideBody(String tenant, long amount) {
         body.put("estimate", Map.of("unit", "TOKENS", "amount", amount));
         return body;
     }
+
+    /**
+     * Seed half the population for the authenticated tenant and half for an unrelated
+     * tenant. The current sorted path scans the global reservation keyspace before
+     * filtering by tenant, so this fixture exposes both population size and tenant
+     * selectivity. Direct pipelined hashes keep setup time out of the measurement.
+     */
+    private void benchmarkSortedReservations(int totalRows, String benchmarkName) {
+        seedReservationPopulation(totalRows);
+
+        long[] timings = runBenchmark(benchmarkName, () -> {
+            ResponseEntity<Map> resp = get(
+                    "/v1/reservations?limit=20&sort_by=created_at_ms&sort_dir=desc",
+                    API_KEY_SECRET_A);
+            assertThat(resp.getStatusCode().value()).isEqualTo(200);
+            assertThat((List<?>) resp.getBody().get("reservations")).hasSize(20);
+        });
+        record(new BenchmarkResult(benchmarkName, p(timings, 50), p(timings, 95), p(timings, 99),
+                timings[0], timings[timings.length - 1], mean(timings)));
+    }
+
+    private void seedReservationPopulation(int totalRows) {
+        final long createdAtBase = 1_700_000_000_000L;
+        try (Jedis jedis = jedisPool.getResource(); Pipeline pipeline = jedis.pipelined()) {
+            for (int i = 0; i < totalRows; i++) {
+                String tenant = i % 2 == 0 ? TENANT_A : TENANT_B;
+                String reservationId = String.format("bench-%05d", i);
+                pipeline.hset("reservation:res_" + reservationId, Map.ofEntries(
+                        Map.entry("reservation_id", reservationId),
+                        Map.entry("tenant", tenant),
+                        Map.entry("state", "ACTIVE"),
+                        Map.entry("subject_json", "{\"tenant\":\"" + tenant + "\"}"),
+                        Map.entry("action_json", "{\"kind\":\"llm.completion\",\"name\":\"benchmark\"}"),
+                        Map.entry("estimate_amount", "100"),
+                        Map.entry("estimate_unit", "TOKENS"),
+                        Map.entry("scope_path", "tenant:" + tenant),
+                        Map.entry("affected_scopes", "[\"tenant:" + tenant + "\"]"),
+                        Map.entry("created_at", String.valueOf(createdAtBase + i)),
+                        Map.entry("expires_at", String.valueOf(createdAtBase + 86_400_000L + i))));
+                if ((i + 1) % 500 == 0) {
+                    pipeline.sync();
+                }
+            }
+            pipeline.sync();
+        }
+    }
 }

cycles-protocol-service/cycles-protocol-service-api/src/test/resources/application-benchmark.properties

@@ -1,6 +1,10 @@
 # Benchmark profile — suppress all logging so only benchmark result tables are visible
 logging.level.root=WARN
 logging.level.io.runcycles.protocol=WARN
+# Large sorted-list fixtures intentionally cross the repository's hydration
+# warning threshold on every measured request. Suppress that expected warning
+# so console I/O does not become part of the latency measurement.
+logging.level.io.runcycles.protocol.data.repository.RedisReservationRepository=ERROR
 logging.level.org.springframework=WARN
 logging.level.org.testcontainers=WARN
 logging.level.com.github.dockerjava=WARN

cycles-protocol-service/pom.xml

@@ -18,7 +18,7 @@
         <module>cycles-protocol-service-api</module>
     </modules>
     <properties>
-        <revision>0.1.25.52</revision>
+        <revision>0.1.25.53</revision>
         <java.version>21</java.version>
         <maven.compiler.source>21</maven.compiler.source>
         <maven.compiler.target>21</maven.compiler.target>

docker-compose.full-stack.prod.yml

@@ -24,7 +24,7 @@ services:
 
   cycles-server:
     logging: *default-logging
-    image: ghcr.io/runcycles/cycles-server:0.1.25.52
+    image: ghcr.io/runcycles/cycles-server:0.1.25.53
     restart: unless-stopped
     ports:
       - "7878:7878"

docker-compose.prod.yml

@@ -24,7 +24,7 @@ services:
 
   cycles-server:
     logging: *default-logging
-    image: ghcr.io/runcycles/cycles-server:0.1.25.52
+    image: ghcr.io/runcycles/cycles-server:0.1.25.53
     restart: unless-stopped
     ports:
       - "7878:7878"