Skip to content

Commit 05e2d17

Browse files
committed
docs: fix encoding spec correctness\n\n- Fix xxHash32 LORO-seed test vectors\n- Document RawTreeMove encoding + usage\n- Remove incorrect serde_columnar row-count claims\n\nCo-authored-by: lody <agent@lody.ai>
1 parent 69a92a8 commit 05e2d17

File tree

3 files changed

+29
-11
lines changed

3 files changed

+29
-11
lines changed

docs/encoding-container-states.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -183,8 +183,6 @@ List container stores ordered elements with element IDs.
183183
┌────────────────────────────────────────────────────────────────────────────┐
184184
│ EncodedListIds (serde_columnar) │
185185
├─────────────────┬──────────────────────────────────────────────────────────┤
186-
│ varint │ Number of elements (N) │
187-
├─────────────────┼──────────────────────────────────────────────────────────┤
188186
│ Column 1 │ peer_idx (DeltaRle encoded usize) │
189187
├─────────────────┼──────────────────────────────────────────────────────────┤
190188
│ Column 2 │ counter (DeltaRle encoded i32) │
@@ -408,8 +406,6 @@ Fractional indexes are prefix-compressed:
408406
┌────────────────────────────────────────────────────────────────────────────┐
409407
│ PositionArena (serde_columnar) │
410408
├─────────────────┬──────────────────────────────────────────────────────────┤
411-
│ varint │ Number of positions (N) │
412-
├─────────────────┼──────────────────────────────────────────────────────────┤
413409
│ Column 1 │ common_prefix_length (Rle encoded usize) │
414410
│ │ Bytes shared with previous position │
415411
├─────────────────┼──────────────────────────────────────────────────────────┤

docs/encoding-xxhash32.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -134,19 +134,19 @@ xxHash32(new Uint8Array([]), 0) === 0x02CC5D05 // 46947589
134134
const LORO_SEED = 0x4F524F4C;
135135

136136
// Empty with LORO seed
137-
xxHash32(new Uint8Array([]), LORO_SEED) === 0x30CFEAB0 // 819588784
137+
xxHash32(new Uint8Array([]), LORO_SEED) === 0xDC3BF95A // 3694917978
138138

139139
// Single byte
140-
xxHash32(new Uint8Array([0x00]), LORO_SEED) === 0x71B1D100 // 1907421440
140+
xxHash32(new Uint8Array([0x00]), LORO_SEED) === 0xDAD9F666 // 3671717478
141141

142142
// "loro" (4 bytes)
143-
xxHash32(new Uint8Array([0x6C, 0x6F, 0x72, 0x6F]), LORO_SEED) === 0x9B07EF77 // 2601328503
143+
xxHash32(new Uint8Array([0x6C, 0x6F, 0x72, 0x6F]), LORO_SEED) === 0x74D321EA // 1959993834
144144

145145
// 16 bytes (triggers block processing)
146146
xxHash32(new Uint8Array([
147147
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
148148
0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F
149-
]), LORO_SEED) === 0xE5AA0AB4 // 3853614772
149+
]), LORO_SEED) === 0x2EDAB25F // 786084447
150150
```
151151

152152
## Usage in Loro Checksum Verification

docs/encoding.md

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -678,7 +678,7 @@ Loro uses a tagged value encoding system where each value is prefixed with a typ
678678
| 4 | F64 | 64-bit floating point (big-endian) |
679679
| 5 | Str | String value (LEB128 len + UTF-8 bytes) |
680680
| 6 | Binary | Binary data (LEB128 len + bytes) |
681-
| 7 | ContainerType | Container type marker (LEB128 index) |
681+
| 7 | ContainerType | Container index (LEB128 unsigned) |
682682
| 8 | DeleteOnce | Single deletion marker |
683683
| 9 | DeleteSeq | Sequence deletion |
684684
| 10 | DeltaInt | Delta-encoded i32 (LEB128 signed) |
@@ -687,7 +687,7 @@ Loro uses a tagged value encoding system where each value is prefixed with a typ
687687
| 13 | TreeMove | Tree node move operation |
688688
| 14 | ListMove | List move operation |
689689
| 15 | ListSet | List set operation |
690-
| 16 | RawTreeMove | Raw tree move (internal) |
690+
| 16 | RawTreeMove | Raw tree move (used for Tree ops in Change Blocks) |
691691
| 0x80+ | Future | Unknown/future value types |
692692

693693
**Source**: `crates/loro-internal/src/encoding/value.rs:39-161`
@@ -788,6 +788,28 @@ Common patterns:
788788
**Source**: `crates/loro-internal/src/encoding/value.rs:480-485` (EncodedTreeMove struct)
789789
**Source**: `crates/loro-internal/src/encoding/value.rs:953-967` (read_tree_move)
790790

791+
#### RawTreeMove
792+
793+
This encoding is used by Change Blocks for Tree operations (Create/Move/Delete).
794+
795+
```
796+
┌──────────────────────────────────────────────────────────────────┐
797+
│ RawTreeMove Encoding │
798+
├───────────────┬──────────────────────────────────────────────────┤
799+
│ LEB128 │ subject_peer_idx (index into peers) │
800+
│ LEB128 │ subject_cnt (TreeID counter) │
801+
│ LEB128 │ position_idx (index into positions) │
802+
│ 1 │ is_parent_null (u8 as bool) │
803+
│ LEB128 │ parent_peer_idx (only if !is_parent_null) │
804+
│ LEB128 │ parent_cnt (only if !is_parent_null) │
805+
└───────────────┴──────────────────────────────────────────────────┘
806+
```
807+
808+
**Source**: `crates/loro-internal/src/encoding/value.rs:470-477` (RawTreeMove struct)
809+
**Source**: `crates/loro-internal/src/encoding/value.rs:969-987` (read_raw_tree_move)
810+
**Source**: `crates/loro-internal/src/encoding/value.rs:1122-1136` (write_raw_tree_move)
811+
**Source**: `crates/loro-internal/src/oplog/change_store/block_encode.rs:316-370` (TreeOp → RawTreeMove)
812+
791813
#### ListMove
792814

793815
```
@@ -1638,7 +1660,7 @@ To implement a complete Loro decoder/encoder, you need to handle:
16381660
- [x] ContainerID encoding/decoding
16391661
- [x] ContainerWrapper encoding/decoding
16401662
- [x] Change Block full parsing
1641-
- [x] ContainerArena encoding/decoding (serde_columnar)
1663+
- [x] ContainerArena encoding/decoding (postcard Vec)
16421664
- [x] PositionArena encoding/decoding (prefix compression)
16431665
- [x] Value encoding/decoding for all types
16441666
- [x] serde_columnar compatible decoder

0 commit comments

Comments
 (0)