perf: cache reconstruction plans and pre-warm on open#21
Merged
Conversation
Contributor
Benchmark Results |
Contributor
POSIX Compliance (pjdfstest) |
Member
Author
Benchmark results (EC2 m6i.2xlarge, sequential read FUSE)
Sequential re-reads are identical (~2300 MB/s both sides, page cache bound). CAS reconstruction calls (PR, total session — 3 files x seq+reread+range+random)
On |
XciD
force-pushed
the
perf/reconstruction-cache
branch
2 times, most recently
from
b30ca81 to
98d25dc
Compare
CachedXetClient now maintains a full-file reconstruction plan cache keyed by MerkleHash with a 59-minute TTL (presigned URLs expire after 1 hour). When a range request misses the cache but the full-file plan is cached, derive_range_response slices the term list locally with no network call. The lock is released before derivation so concurrent pread() calls do not serialize on it. Cache entries store only full-file plans (no per-range entries), which eliminates stale range responses and keeps the cache small.
warm_reconstruction_cache() fires a background tokio task at open() that fetches the full-file reconstruction plan from CAS. A watch::Receiver signals completion; the first read() awaits it only if the warmup is not yet done. For workloads with any gap between open() and the first read (safetensors header parsing, Python overhead), the CAS round-trip overlaps with userspace work and adds zero latency to reads. FUSE open replies with FOPEN_KEEP_CACHE so the kernel page cache is reused across re-opens of the same file.
XciD
force-pushed
the
perf/reconstruction-cache
branch
from
98d25dc to
f8172fe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two complementary optimizations to eliminate redundant CAS API round-trips for reconstruction plan lookups.
derive_range_responseinCachedXetClientWhen a range-query (
get_reconstruction(hash, Some(range))) misses the cache but the full-file plan (get_reconstruction(hash, None)) is already cached, derive the range response locally by slicing the term list -- no network call. The mutex is released before the derivation so N concurrentpread()calls don't serialize on it.warm_reconstruction_cacheatopen()A fire-and-forget task fetches the full-file reconstruction plan at
open()time. By the time the firstread()arrives, the plan is typically already in the in-memory cache. Errors are silently dropped (best-effort).Together these mean that after the first
open(), all reconstruction lookups for that file are served from memory.