Skip to content

test: xorb reconstruction cache warm-up benchmark and unit test#24

Merged
XciD merged 1 commit into
mainfrom
perf/warm-cache-test
Mar 8, 2026
Merged

test: xorb reconstruction cache warm-up benchmark and unit test#24
XciD merged 1 commit into
mainfrom
perf/warm-cache-test

Conversation

@XciD

@XciD XciD commented Mar 8, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds a unit test verifying warm_reconstruction_cache is triggered on open() and that the watch stays true for subsequent reads on the same handle (no re-waiting)
  • Adds an EC2 integration benchmark (warm_cache_bench) measuring TTFB and sequential throughput for 5 consecutive opens of a 100 MB file

Background

The streaming path already used the xorb chunk disk cache correctly via FileReconstructor.with_chunk_cache(). This PR adds validation and numbers to prove it.

Benchmark results (m6i.2xlarge, /dev/shm cache)

Open#  TTFB (ms)  Total (s)  MB/s     cache
-----  ---------  ---------  -------  -----
    1      200.7       0.23    432.0  cold
    2        0.8       0.02   5741.3  warm
    3        0.7       0.01   6789.3  warm
    4        0.6       0.01   7050.2  warm
    5        0.6       0.01   7040.3  warm

cold TTFB 201 ms / 432 MB/s  →  warm TTFB 1 ms / 6.7 GB/s

The bench uses /dev/shm (tmpfs) on Linux so warm reads hit RAM. On EBS-backed /tmp (m6i default, ~125 MB/s baseline), warm reads would appear slower than cold S3 reads, making the cache benefit invisible.

Run

cargo test --release --test warm_cache_bench -- --nocapture

Add a unit test verifying that warm_reconstruction_cache is triggered on
open() and that subsequent reads on the same handle are served without
re-waiting (watch value stays true).

Add an EC2 integration benchmark (warm_cache_bench) that measures TTFB and
sequential throughput for 5 consecutive opens of a 100 MB file:
- Open 1 (cold): CAS round-trip for reconstruction plan
- Open 2+ (warm): plan served from in-process cache, data from xorb cache

Use /dev/shm (tmpfs) as the xorb cache directory on Linux so warm reads hit
RAM rather than EBS, demonstrating the actual benefit:

  cold  TTFB 201 ms / 432 MB/s
  warm  TTFB   1 ms / 6.7 GB/s
@github-actions

github-actions Bot commented Mar 8, 2026

Copy link
Copy Markdown
Contributor

POSIX Compliance (pjdfstest)

============================================================
  pjdfstest POSIX Compliance Results
------------------------------------------------------------
  Files: 130/130 passed    Tests: 832 total (0 subtests failed)
  Result: PASS
------------------------------------------------------------
  Category               Passed    Total   Status
  -------------------- -------- -------- --------
  chflags                     5        5       OK
  chmod                       8        8       OK
  chown                       6        6       OK
  ftruncate                  13       13       OK
  granular                    5        5       OK
  mkdir                       9        9       OK
  open                       19       19       OK
  posix_fallocate             1        1       OK
  rename                     10       10       OK
  rmdir                      11       11       OK
  symlink                    10       10       OK
  truncate                   13       13       OK
  unlink                     11       11       OK
  utimensat                   9        9       OK
============================================================

@github-actions

github-actions Bot commented Mar 8, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

============================================================
  Benchmark — 50MB
------------------------------------------------------------
  Metric                                 FUSE          NFS
  ------------------------------ ------------ ------------
  Sequential read                    193.1 MB/s     191.8 MB/s
  Sequential re-read                2252.8 MB/s    2276.7 MB/s
  Range read (1MB@25MB)                0.4 ms         0.2 ms
  Random reads (100x4KB avg)           0.0 ms         0.0 ms
  Sequential write (FUSE)           1033.3 MB/s
  Close latency (CAS+Hub)            0.132 s
  Write end-to-end                   277.9 MB/s
  Dedup write                       1037.1 MB/s
  Dedup close latency                0.069 s
  Dedup end-to-end                   428.0 MB/s
============================================================
============================================================
  Benchmark — 200MB
------------------------------------------------------------
  Metric                                 FUSE          NFS
  ------------------------------ ------------ ------------
  Sequential read                   1379.9 MB/s    1254.5 MB/s
  Sequential re-read                2209.6 MB/s    2379.9 MB/s
  Range read (1MB@25MB)                0.2 ms         0.2 ms
  Random reads (100x4KB avg)           0.0 ms         0.0 ms
  Sequential write (FUSE)           1009.0 MB/s
  Close latency (CAS+Hub)            1.665 s
  Write end-to-end                   107.3 MB/s
  Dedup write                       1078.9 MB/s
  Dedup close latency                0.142 s
  Dedup end-to-end                   611.5 MB/s
============================================================
============================================================
  Benchmark — 500MB
------------------------------------------------------------
  Metric                                 FUSE          NFS
  ------------------------------ ------------ ------------
  Sequential read                   1595.7 MB/s    1544.1 MB/s
  Sequential re-read                2170.5 MB/s    2422.8 MB/s
  Range read (1MB@25MB)                0.2 ms         0.2 ms
  Random reads (100x4KB avg)           0.0 ms         0.0 ms
  Sequential write (FUSE)            989.3 MB/s
  Close latency (CAS+Hub)            1.734 s
  Write end-to-end                   223.3 MB/s
  Dedup write                       1027.7 MB/s
  Dedup close latency                0.100 s
  Dedup end-to-end                   852.9 MB/s
============================================================
============================================================
  fio Benchmark Results
------------------------------------------------------------
  Job                        FUSE MB/s   NFS MB/s  FUSE IOPS   NFS IOPS
  ------------------------- ---------- ---------- ---------- ----------
  seq-read-100M                  505.0      286.5                      
  seq-reread-100M               1923.1     1408.5                      
  rand-read-4k-100M                0.4        0.4         98        114
  seq-read-5x10M                 574.7      312.5                      
  rand-read-10x1M                173.5       62.5      44428      16002
  Random Read Latency           FUSE avg      NFS avg
  ------------------------- ------------ ------------
  rand-read-4k-100M           10149.7 us    8756.3 us
  rand-read-10x1M                21.8 us      61.6 us
============================================================

@XciD XciD marked this pull request as ready for review March 8, 2026 12:59
@XciD XciD merged commit 20ccab1 into main Mar 8, 2026
4 checks passed
@XciD XciD deleted the perf/warm-cache-test branch March 23, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant