Skip to content

Commit 37a8e0e

Browse files
committed
Add opt-in Snappy page compression to reduce database size
Introduce a `Compression` option on DB/Options that enables transparent Snappy compression of leaf and branch page data. Compression happens at node spill time—before page allocation—so fewer pages are allocated for compressible data. Decompression is transparent on read via a per-transaction cache using sync.Map for concurrent reader safety. Key changes: - New orthogonal CompressedPageFlag (0x20) on page headers; type checks changed from == to bitwise & so the flag coexists with page types - CompressInodes serializes and compresses node data, only used when it reduces the page count - Split threshold increased 4x when compression is enabled so nodes accumulate enough data for meaningful compression - DecompressPage preserves on-disk overflow for correct freelist accounting - Adds github.com/golang/snappy dependency This was largely written by Claude Opus. Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
1 parent 39fc24c commit 37a8e0e

File tree

11 files changed

+987
-17
lines changed

11 files changed

+987
-17
lines changed

bucket.go

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -884,6 +884,11 @@ func (b *Bucket) node(pgId common.Pgid, parent *node) *node {
884884
})
885885
}
886886

887+
// If the page is compressed, decompress it before reading.
888+
if p.IsCompressed() {
889+
p = b.tx.decompressedPage(p)
890+
}
891+
887892
// Read the page into the node and cache it.
888893
n.read(p)
889894
b.nodes[pgId] = n
@@ -945,7 +950,14 @@ func (b *Bucket) pageNode(id common.Pgid) (*common.Page, *node) {
945950
}
946951

947952
// Finally lookup the page from the transaction if no node is materialized.
948-
return b.tx.page(id), nil
953+
p := b.tx.page(id)
954+
955+
// If the page is compressed, decompress it transparently.
956+
if p.IsCompressed() {
957+
p = b.tx.decompressedPage(p)
958+
}
959+
960+
return p, nil
949961
}
950962

951963
// BucketStats records statistics about resources used by a bucket.

0 commit comments

Comments
 (0)