Merge pull request #203 from JairusSW/jairus/v1.5.0-prep

fix(dynamic): nullable-string null + dynamic-string/lazy-slot perf, c…

Author: JairusSW

CHANGELOG.md

@@ -2,7 +2,7 @@
 
 ## Unreleased
 
-## 2026-06-09 - v1.5.0
+## 2026-06-10 - v1.5.0
 
 - perf(dynamic): **`JSON.Obj`, `JSON.Value`, and `JSON.Arr` are now lazy by default** - a near-alloc-less, simdjson On-Demand-style rework of dynamic parsing. `JSON.parse<JSON.Obj>` no longer eagerly materializes the whole tree: each nested value stores its raw source slice and is parsed only on first access (`.get<T>()` / `.getAs<T>()` / `.at(i)`), then cached - a value you never read is never parsed, and an untouched value re-serializes by copying its original source bytes verbatim. `JSON.Obj` is backed by a `StaticArray<u64>` value-slot buffer plus a length-prefixed key buffer (keys emit straight from their slice - no per-key string materialization), and a new buffer-backed `JSON.Arr` mirrors it (`.at(i)` → `JSON.Value`, `.getAs<T>(i)`, `.push<T>`, `.set<T>`, `.length`). Deferred composites reuse the NaN-boxed `JSON.Value` slot, and the SIMD/SWAR value scanners gained a vectorized composite (`{}`/`[]`) scan. Net for proxy / filter / forward workloads over large payloads: dynamic deserialize is several× faster with far fewer allocations, and untouched round-trips are byte-exact passthrough. A new `dynamic-interop` suite covers `Map`, `Date`, `JSON.Box`, `JSON.Raw`, and nested-struct interop
 - perf(`JSON.Obj`): **faster dynamic key access.** `indexOf` now linear-scans objects with ≤ 6 keys (the common case) instead of allocating and hashing a key index - small objects pay no index build/probe cost, and a dynamic-access workload that touches only a few keys never builds an index it won't reuse; larger objects keep the open-addressed hash index. The per-lookup key comparison (`utf16Equals`, shared by the linear scan and the hash probe) is widened to 8 code units per step on the SIMD build (one `v128`) and 4 per step on naive/swar (one `u64`), each load bounded by the key length so it never over-reads. Warm long-key lookups **+37–133%**, short keys **+10–20%**, no cold-path change
@@ -16,6 +16,11 @@
 - fix(bench): restore the lazy **access-pattern** benchmark dropped when `lazy.bench.ts` was split into per-concern files. `assembly/__benches__/lazy/access-pattern.bench.ts` re-emits the `lz-access` suite (eager read-all baseline + lazy read-none / read-one / read-all / passthrough on a medium struct) that `scripts/build-chart15.ts` reads for `lazy-access-pattern.svg` - the chart had been aborting with ENOENT on `lz-access.eager`
 - tooling: `bun run playground:tmp` builds and runs the transform-generated `assembly/playground.tmp.ts` directly (no transform) under the v8 bench runner, for hand-tuning the generated codec. `assembly/playground.ts` is now a fast-path (non-lazy, eager) deserialize micro-bench - a direct `__DESERIALIZE_FAST` into a reused object with a min-over-rounds timer
 - tooling: `npm run bench:all` (`scripts/bench-all.sh`) runs the full benchmark matrix in one shot - root files plus `multilib/`, `throughput/`, `prim/` (AS + JS) and `classic/`, `lazy/` (AS-only) - forwarding flags (`--mode`, `--v8`, `--wavm`) to the AS runner and continuing past a failing category (non-zero exit if any failed). `npm run charts:publish` (`scripts/publish-benchmarks.sh`) now publishes by default even with a dirty/untracked working tree - chart output only ever commits to a separate `docs` worktree, never your main tree - with `PUBLISH_REQUIRE_CLEAN=1` to restore the old refuse-if-dirty guard
+- fix(lazy/dynamic): a `null` value for a **nullable string** field on the lazy/dynamic path was mis-decoded. `JSON.__deserialize` (used by lazy-field materialization, `JSON.parse<JSON.Value>`, and `JSON.Obj`/`Arr` value slots) tested `isString<T>()` - which is `true` for `string | null` - before the `null`-literal check, so a bare `null` was parsed as a quoted string: an abort (`Invalid JSON string: missing surrounding quotes`) under NAIVE, silent garbage under SWAR/SIMD. The `null` check now precedes the string branch, matching the eager `parseInternal` path. Surfaced as a crash materializing absent-as-`null` string fields in the `classic/citm_catalog.lazy` benchmark
+- perf(dynamic): **re-serializing a dynamic string emits via `memory.copy` when it needs no escaping.** A materialized `JSON.Value` / `JSON.Obj` / `JSON.Arr` string caches a 2-bit escape class in two otherwise-unused bits of its NaN-box payload (a wasm32 pointer is 32 of the 45 payload bits); it's classified once on first serialize, then reused - AS strings are immutable, so the verdict never goes stale. A clean string then round-trips as `"` + one `memory.copy` + `"` instead of a per-character escape scan: **~3×** faster re-serialize (3.8 → 11.5 GB/s on a 1 MB string). `JSON.Obj`/`Arr` persist the class back into their flat value slot so the win carries across re-serializes; typed struct `string` fields (no box to cache in) are unchanged
+- perf(dynamic): rebalanced the lazy value-slot packing from **22/22 to 23/21 bits** (offset/length). Object/array fields are usually small while the document can be large, so offset overflow - a field late in a big doc - is the realistic trigger; widening the offset field lifts the compact (no-rescan) range from ~8 MB to **~16 MB** of source, at the cost of single field values over ~4 MB falling back to the existing absolute (scan-on-demand) slot form
+- test(dynamic): `dynamic-string-class` (clean / escaped / surrogate / empty strings round-tripped through `JSON.Value` and re-serialized, plus `JSON.Obj`/`Arr` slot-class caching) and `lazy-slot-encoding` (the packed slot's compact↔absolute boundary, previously untested) suites
+- bench/docs: benchmark chart scripts renamed from `chartNN` to descriptive names (`overview-` / `string-` / `object-` / `primitive-` / `library-{serialize,deserialize}`); every chart now emits both SVG and PNG; the README switched fully to SVG with the real-world payload charts promoted to the top of the Performance section and a "browse the full chart set" link that the publish script re-pins per release. String throughput charts gained a `JSON.Value` (dynamic) series, and the classic charts got vertical value labels
 
 ## 2026-06-05 - v1.4.0
 

README.md

@@ -17,10 +17,11 @@
   - [Using Custom Serializers or Deserializers](#using-custom-serializers-or-deserializers)
   - [Overriding built-in Container Types](#overriding-built-in-container-types)
 - [Performance](#performance)
+  - [Real-World Throughput](#real-world-throughput)
   - [Comparison to JavaScript](#comparison-to-javascript)
+  - [Library Comparison](#library-comparison)
+  - [Lazy Fields](#lazy-fields)
   - [Performance Tuning](#performance-tuning)
-  - [Fast-Path Compatibility Matrix](#fast-path-compatibility-matrix)
-  - [Container Compatibility Matrix](#container-compatibility-matrix)
   - [Running Benchmarks Locally](#running-benchmarks-locally)
 - [Debugging](#debugging)
 - [Architecture](#architecture)
@@ -505,79 +506,103 @@ This same pattern works for subclassable built-ins like `Array`, `Map`, `Set`, a
 
 ## Performance
 
-The `json-as` library is engineered for **multi-GB/s processing speeds**, leveraging SIMD and SWAR optimizations along with highly efficient transformations. The charts below highlight key performance metrics such as build time, operations-per-second, and throughput.
+`json-as` is engineered for **multi-GB/s** serialization and deserialization. Every `@json` schema is compiled to specialized WebAssembly at build time, and bytes are scanned by one of three interchangeable backends:
 
-### Comparison to JavaScript
+- **NAIVE** — a portable, branchy scalar scanner. The correctness baseline; needs no special CPU features.
+- **SWAR** — *SIMD-Within-A-Register*: processes 8 bytes at a time with ordinary 64-bit integer math. The default.
+- **SIMD** — true 128-bit vector scanning. Fastest on large and string-heavy payloads; enable with `--enable simd`.
 
-The following charts compare JSON-AS against JavaScript's native `JSON` implementation. It's as fair as possible and runs on V8's turboshaft optimizer. The published charts are generated locally and pushed to the `docs` branch.
+The mode is chosen per build via `JSON_MODE` (see [Performance Tuning](#performance-tuning)). Orthogonal to the scan mode, the generated **struct** path can be swapped for **[lazy](#lazy-fields)** fields (defer parsing until first access) or the fully dynamic, schema-less **`JSON.Obj`** path.
 
-> Note: Benchmarks reflect the **latest version**. Older versions may show different performance.
->
-> Current local benchmark machine: Apple M4 Max (16 cores - 12 performance + 4 efficiency), 64 GB RAM, macOS 26.
+> All figures below are **end-to-end**: deserialization includes allocating the destination object/array, not just scanning bytes — raw parser throughput is higher. Charts are generated locally and pushed to the [`docs`](https://github.com/JairusSW/json-as/tree/docs) branch, and reflect the **latest release** (older versions may differ).
 >
-> Benchmark results include normal end-to-end work such as allocating the destination object or array before deserializing into it. Raw parser throughput is higher than the published figures because these numbers intentionally include that allocation/setup cost.
+> Benchmark machine: AMD Ryzen 7 7800X3D (8 cores, 96 MB 3D V-Cache), 32 GB RAM, Zorin OS 18.1 (Linux 6.17). JavaScript baselines run on V8's turboshaft optimizer.
+
+📊 **[Browse the full chart set for this release →](https://github.com/JairusSW/json-as/tree/docs/charts/v1.5.0)**
+
+### Real-World Throughput
+
+The headline benchmark: nine standard JSON payloads — drawn from the [yyjson](https://github.com/ibireme/yyjson) and [`nativejson-benchmark`](https://github.com/miloyip/nativejson-benchmark) corpora — measured in all three scan modes, with the SIMD lazy-struct and dynamic `JSON.Obj` paths shown alongside.
+
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/classic-payload-deserialize-v8.svg" alt="Deserialization throughput across nine classic JSON payloads">
+
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/classic-payload-serialize-v8.svg" alt="Serialization throughput across nine classic JSON payloads">
+
+Each payload stresses a different document shape:
+
+| Payload | Size | What it stresses |
+|---------|------|------------------|
+| **Twitter** | 467 KB | API response, fully-modeled struct schema with `@optional` keys |
+| **Canada** | 2.1 MB | GeoJSON — deeply nested arrays of floating-point coordinates |
+| **CITM** | 500 KB | Concert catalog — dynamic-key `Map`s plus uniform struct arrays |
+| **Poet** | 3.3 MB | ~8,900 flat `{desc, name, id}` records — pure struct fast path |
+| **GitHub** | 53 KB | 30 GitHub events — a wide union of per-event-type fields |
+| **GSOC** | 3.1 MB | ~1,264 org records keyed by id (schema.org JSON-LD `Map`) |
+| **Lottie** | 289 KB | Vector-animation doc — structs over deeply variable layer data |
+| **otfcc** | 66.4 MB | OpenType font dump — 15 tables captured as `JSON.Raw` |
+| **FGO** | 48.8 MB | Game-data dump — 193 irregular tables as `Map<string, JSON.Raw>` |
+
+The five series in each chart:
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart01.svg" alt="Performance Chart 1">
+- **NAIVE / SWAR / SIMD** — the generated struct path under each scan backend, parsing every field into a typed schema.
+- **Lazy (SIMD)** — `@json({ lazy: "auto" })`: each field's raw slice is stored at parse time and decoded only on first access; on serialize, untouched fields stream their original bytes straight back out. Reading or rewriting a subset is dramatically cheaper — see [Lazy Fields](#lazy-fields).
+- **JSON.Obj (SIMD)** — a fully dynamic, schema-less parse into `JSON.Obj`, with no generated per-type code.
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart02.svg" alt="Performance Chart 2">
+### Comparison to JavaScript
+
+`json-as` against JavaScript's native `JSON`, parsed and stringified in a fresh V8 — as close to apples-to-apples as a Wasm-vs-native comparison gets.
+
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/overview-serialize.svg" alt="Performance Chart 1">
+
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/overview-deserialize.svg" alt="Performance Chart 2">
 
 <details>
 <summary>String serialize charts (click to expand)</summary>
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart03.png" alt="Performance Chart 3">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/string-serialize.svg" alt="Performance Chart 3">
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart07.png" alt="Performance Chart 7">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/string-serialize-1mb.svg" alt="Performance Chart 7">
 </details>
 
 <details>
 <summary>String deserialize charts (click to expand)</summary>
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart04.png" alt="Performance Chart 4">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/string-deserialize.svg" alt="Performance Chart 4">
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart08.png" alt="Performance Chart 8">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/string-deserialize-1mb.svg" alt="Performance Chart 8">
 </details>
 
 <details>
 <summary>Object serialize charts (click to expand)</summary>
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart05.png" alt="Performance Chart 5">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/object-serialize.svg" alt="Performance Chart 5">
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart09.png" alt="Performance Chart 9">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/object-serialize-1mb.svg" alt="Performance Chart 9">
 </details>
 
 <details>
 <summary>Object deserialize charts (click to expand)</summary>
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart06.png" alt="Performance Chart 6">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/object-deserialize.svg" alt="Performance Chart 6">
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart10.png" alt="Performance Chart 10">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/object-deserialize-1mb.svg" alt="Performance Chart 10">
 </details>
 
 <details>
 <summary>Primitive (de)serialize charts (click to expand)</summary>
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart11.svg" alt="Primitive serialization performance">
-
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart12.svg" alt="Primitive deserialization performance">
-</details>
-
-<details>
-<summary>Real-world payloads — eager / lazy / dynamic (click to expand)</summary>
-
-Throughput across nine classic JSON payloads (Twitter, Canada, CITM, Poet, GitHub events, GSOC, Lottie, otfcc, FGO), each in the NAIVE / SWAR / SIMD scan modes, with the SIMD lazy-struct and dynamic `JSON.Obj` paths shown alongside.
-
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/classic-payload-deserialize-v8.png" alt="Classic payload deserialize throughput">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/primitive-serialize.svg" alt="Primitive serialization performance">
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/classic-payload-serialize-v8.png" alt="Classic payload serialize throughput">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/primitive-deserialize.svg" alt="Primitive deserialization performance">
 </details>
 
-### Library comparison
+### Library Comparison
 
 How `json-as` stacks up against other JSON libraries on a ~5 KiB GitHub-repo payload: JavaScript's native `JSON` and `fast-json` (each in a fresh V8), plus the `assemblyscript-json` package. The `json-as` bars (generated struct, lazy struct, and dynamic `JSON.Obj`) are averaged across the NAIVE / SWAR / SIMD scan modes.
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart14.png" alt="Library comparison - deserialize throughput">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/library-deserialize.svg" alt="Library comparison - deserialize throughput">
 
-<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/chart13.png" alt="Library comparison - serialize throughput">
+<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/library-serialize.svg" alt="Library comparison - serialize throughput">
 
 ### Lazy Fields
 

assembly/__benches__/lazy/access-pattern.bench.ts

@@ -12,7 +12,7 @@ import { bench, blackbox, dumpToFile, utf8ByteLength } from "../lib/bench";
 // Fields are marked deferrable with explicit `@lazy` (per the maintainer's
 // request) rather than class-level `lazy: "auto"`, so each payload has a known
 // deferred-field count regardless of the auto threshold. The dedicated chart
-// (build-chart15.ts) reads the SWAR logs only - lazy is showcased in SWAR.
+// (build-lazy.ts) reads the SWAR logs only - lazy is showcased in SWAR.
 //
 // Dumps: lzap-<payload>.{base,none,one,half,all}.