fix(encode)!: canonically sort major type 7 map keys - #186
Conversation
|
Thanks for picking this up (tbh I haven't cared much about non-string keys), and it's a nice start, but this still isn't RFC 7049 (3.9) ordering for float keys since we're aiming for that. The rule is: encoded length first, then bytewise lexical order; numeric value isn't equivalent. What we have here currently is:
So we need to make the major-7 comparison equivalent to encoded length followed by encoded bytes, honouring Also we should Also fwiw RFC's 8949 restated rules end up having major-7 sorted in the same way, it's just defined differently:
For major-7 we end up in the same place anyway, so nice and consistent and easy in this case. |
ff012a9 to
c610e95
Compare
|
Reworked to order major 7 keys by their canonical encoding rather than by value.
It honours Tests cover negative floats, mixed widths, NaN and the float64 case in both insertion orders. Full suite green. |
c610e95 to
497e7d9
Compare
|
force pushed a rebased branch plus a caching fix over the top (497e7d9); for plain floats I get a speed up of 1.5x for a map with 10 entries, nearly 4x for a map with 10k entries (!). for |
There was a problem hiding this comment.
🟢 Ready to approve
The change addresses a concrete determinism bug, implements canonical bytewise ordering for major-7 keys, and includes targeted regression tests covering key edge cases.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR fixes non-deterministic CBOR map key ordering for “major type 7” keys (simple values and floats), ensuring canonical, insertion-order-independent encoding.
Changes:
- Updates the default map sorter to route same-major comparisons through the major encoder’s
compareTokens(instead of falling back toType.compare). - Implements a major-7 comparator that orders keys by canonical encoded bytes (length-first, then lexicographic), honoring
options.float64. - Adds regression tests covering simple-value and float map key ordering (including NaN, mixed widths, and
float64mode).
File summaries
| File | Description |
|---|---|
lib/encode.js |
Fixes map sorting to compare by major type first, then defer to per-major compareTokens, and passes options into comparators. |
lib/7float.js |
Implements canonical major-7 key comparison via encoded-byte generation + caching, honoring float64. |
interface.ts |
Extends TokenTypeEncoder.compareTokens and MapSorter signatures to optionally accept EncodeOptions. |
test/test-5map.js |
Adds tests asserting deterministic ordering for simple values and floats (including NaN and float64). |
types/interface.d.ts |
Updates public type declarations to match the new comparator/sorter signatures. |
types/interface.d.ts.map |
Regenerates sourcemap for updated interface declarations. |
types/lib/7float.d.ts |
Updates float encoder type declarations (including compareTokens signature) and adds a helper token type for cached bytes. |
types/lib/7float.d.ts.map |
Regenerates sourcemap for updated float declarations. |
types/lib/encode.d.ts.map |
Regenerates sourcemap reflecting updated encode-side typing/structure. |
Review details
- Files reviewed: 4/9 changed files
- Comments generated: 1
- Review effort level: Low
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
Sort simple values and floats by their RFC 7049 canonical encodings, honouring float64, so map encoding no longer depends on insertion order. This changes encoded output for maps containing simple-value or floating-point keys that previously retained insertion order. BREAKING CHANGE: Map encodings containing simple-value or floating-point keys may change byte order. Hashes, signatures, and identifiers derived from affected encodings will therefore change.
Cache encoded key bytes while sorting to avoid repeated conversion.
497e7d9 to
be82627
Compare
|
🎉 This PR is included in version 6.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
The default map key sorter compared key types with
keyToken1.type !== keyToken2.typeand fell back toType.compare, which only looks at the major type. false, true, null, undefined and floats all share major type 7, so any pair of them compared as equal and kept their insertion order. A map keyed by these values encoded to different bytes depending on insertion order, and pairs such as true/false came out in the wrong order (true0xf5before false0xf4).This routes keys that share a major type through the type encoder's
compareTokensand gives the major 7 encoder a comparator that orders the simple values by their encoded byte and floats by value, so these keys sort deterministically. Adds tests for simple value and float map keys.