Skip to content

Commit 16ec2e9

Browse files
benasher44claude
andcommitted
README: real native-vs-WASM comparison with measured numbers
Replace the native-only allocator table with a 3-way comparison (WASM, native system malloc, native jemalloc) using isolated per-process measurements: WASM peaks at 1359 MB and never releases (+1202 MB retained), native+jemalloc peaks at 498 MB and stabilizes (+377 MB), and native is ~11% faster on throughput. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent a1e5192 commit 16ec2e9

1 file changed

Lines changed: 26 additions & 18 deletions

File tree

β€Žnative/README.mdβ€Ž

Lines changed: 26 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -15,26 +15,34 @@ actually be returned to the OS. Pairing it with **jemalloc** (via `LD_PRELOAD` /
1515
`DYLD_INSERT_LIBRARIES`) roughly halves peak RSS and, more importantly, keeps it
1616
*stable* across repeated large parses instead of ratcheting.
1717

18-
### Measured: native, system malloc vs jemalloc
18+
### Measured: native vs WASM
1919

2020
Parsing a 3.31 MB SQL query (1500Γ— `UNION ALL`, ~65 MB JSON parse tree), 3 parse/free
21-
cycles, `darwin-arm64`, Node 24. Peak is the max RSS during a parse; "after" is RSS
22-
once the result is dropped and GC settles.
23-
24-
| Allocator | cycle 1 peak | cycle 2 peak | cycle 3 peak | retained after |
25-
|-----------|-------------|-------------|-------------|----------------|
26-
| system malloc | 649 MB | 867 MB | **932 MB** | 867 MB |
27-
| jemalloc | 381 MB | 497 MB | **498 MB** | 433 MB |
28-
| jemalloc (`dirty_decay_ms:0,muzzy_decay_ms:0`) | 377 MB | 477 MB | **477 MB** | 411 MB |
29-
30-
System malloc climbs every cycle (649 β†’ 932 MB) β€” it fragments and holds the freed
31-
arenas. jemalloc stabilizes at ~498 MB by cycle 2 and stays there. Throughput is
32-
identical either way (~140k small-query parses/sec on this machine), so jemalloc is a
33-
pure memory win with no speed cost.
34-
35-
Reproduce with `bash benchmark/compare-allocators.sh --cycles 3`. A side-by-side against
36-
the WASM backend is available via `node --expose-gc benchmark/memory.mjs --all` once
37-
`@libpg-query/parser` is installed.
21+
cycles, `darwin-arm64`, Node 24. Each backend measured in its own process. "Retained"
22+
is RSS after the result is dropped and GC settles; throughput is a small query Γ—10k.
23+
24+
| Backend | idle RSS | peak RSS (max of 3) | retained after free | throughput |
25+
|---------|---------|---------------------|---------------------|------------|
26+
| WASM (`@libpg-query/parser`) | 93 MB | 1359 MB | **+1202 MB** (never shrinks) | 125k/s |
27+
| Native β€” system malloc | 53 MB | 932 MB | +812 MB (ratchets up) | 139k/s |
28+
| Native β€” **jemalloc** | 55 MB | **498 MB** | **+377 MB** (stabilizes) | 139k/s |
29+
30+
Per-cycle peak progression:
31+
32+
```
33+
WASM: 1261 β†’ 1359 β†’ 1359 MB (plateaus at a high permanent floor)
34+
Native system: 649 β†’ 867 β†’ 932 MB (fragments, still climbing)
35+
Native jemalloc: 381 β†’ 497 β†’ 498 MB (flat after cycle 2)
36+
```
37+
38+
WASM linear memory only ever grows, so ~1.2 GB from one big parse is held for the
39+
process lifetime. Native + system malloc is lower but still ratchets. Native +
40+
jemalloc has ~2.7Γ— lower peak than WASM, returns freed pages to the OS, and stabilizes.
41+
`MALLOC_CONF=dirty_decay_ms:0,muzzy_decay_ms:0` trims peak a little further (~477 MB).
42+
Throughput is identical across allocators β€” jemalloc is a pure memory win.
43+
44+
Reproduce with `node --expose-gc benchmark/memory.mjs --all --cycles 3` (with
45+
`@libpg-query/parser` installed) and `bash benchmark/compare-allocators.sh --cycles 3`.
3846

3947
## Installation
4048

0 commit comments

Comments
Β (0)