Skip to content

Commit 69a92a8

Browse files
committed
docs: fix MovableList semantics and JS examples
- Fix MovableListState invisible_list_item semantics: items follow the visible item (AFTER), not before. Updated description and decoding logic to match source code (line 1509 shows incrementing previous record's counter) - Fix MapState JS example: properly advance past peer-count varint before reading peer table, and correctly slice remaining bytes for per-key metadata decoding - Add Unicode awareness note to RichtextState: span.len is Unicode scalar count, not UTF-16 code units; String.slice() will fail for non-BMP characters - Add counter case (type 5) to Complete Decoding Example switch
1 parent a9aa7b9 commit 69a92a8

File tree

1 file changed

+35
-13
lines changed

1 file changed

+35
-13
lines changed

docs/encoding-container-states.md

Lines changed: 35 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -114,30 +114,35 @@ Map container stores key-value pairs with CRDT metadata.
114114

115115
```javascript
116116
function decodeMapState(bytes) {
117-
let offset = 0;
118-
119117
// 1. Decode visible map values
120118
const [mapValue, rest1] = postcard.takeFromBytes(bytes);
121119

122120
// 2. Decode keys with None values
123121
const [keysWithNone, rest2] = postcard.takeFromBytes(rest1);
124122

125123
// 3. Decode peer table
126-
const peerCount = decodeLEB128(rest2);
124+
let offset = 0;
125+
const [peerCount, peerCountBytes] = decodeLEB128WithSize(rest2);
126+
offset += peerCountBytes; // Advance past the varint
127+
127128
const peers = [];
128129
for (let i = 0; i < peerCount; i++) {
129130
peers.push(readU64LE(rest2, offset));
130131
offset += 8;
131132
}
132133

133-
// 4. Decode per-key metadata
134+
// 4. Decode per-key metadata (from remaining bytes after peer table)
135+
let metaBytes = rest2.slice(offset);
136+
134137
// Keys from both mapValue and keysWithNone, sorted alphabetically
135138
const allKeys = [...Object.keys(mapValue), ...keysWithNone].sort();
136139

137140
const entries = [];
138141
for (const key of allKeys) {
139-
const peerIdx = decodeLEB128(rest2);
140-
const lamport = decodeLEB128(rest2);
142+
const [peerIdx, peerIdxBytes] = decodeLEB128WithSize(metaBytes);
143+
metaBytes = metaBytes.slice(peerIdxBytes);
144+
const [lamport, lamportBytes] = decodeLEB128WithSize(metaBytes);
145+
metaBytes = metaBytes.slice(lamportBytes);
141146
entries.push({
142147
key,
143148
value: keysWithNone.includes(key) ? null : mapValue[key],
@@ -320,6 +325,13 @@ function decodeRichtextState(bytes) {
320325
}
321326
```
322327

328+
> **Note on Unicode**: The `span.len` field represents Unicode scalar count (Rust's
329+
> `unicode_len()`), not byte length or UTF-16 code units. The JavaScript example uses
330+
> `String.slice()` which operates on UTF-16 code units, so it will produce incorrect
331+
> results for text containing characters outside the BMP (e.g., emoji, rare CJK).
332+
> Implementers should use proper Unicode scalar iteration (e.g., `[...str]` spread
333+
> or `Intl.Segmenter` for grapheme-aware handling).
334+
323335
**Source**: `crates/loro-internal/src/state/richtext_state.rs:1130-1339`
324336

325337
---
@@ -453,20 +465,29 @@ Movable list allows elements to be moved while preserving their identity.
453465
┌────────────────────────────────────────────────────────────────────────────┐
454466
│ Column │ Strategy │ Description │
455467
├───────────────────────┼───────────┼────────────────────────────────────────┤
456-
│ invisible_list_item │ DeltaRle │ Count of invisible items before this │
468+
│ invisible_list_item │ DeltaRle │ Count of invisible items AFTER this
457469
│ pos_id_eq_elem_id │ BoolRle │ True if position ID == element ID │
458470
│ elem_id_eq_last_set_id│ BoolRle │ True if element ID == last set ID │
459471
└───────────────────────┴───────────┴────────────────────────────────────────┘
460472
```
461473

462474
### Decoding Logic
463475

464-
The first item in `items` is a sentinel (skip it). For subsequent items:
465-
1. If `invisible_list_item > 0`, consume that many invisible position IDs first
466-
2. Consume one visible position from `list_item_ids`
467-
3. If `pos_id_eq_elem_id` is false, consume from `elem_ids`; otherwise, elem_id = position_id.idlp()
468-
4. If `elem_id_eq_last_set_id` is false, consume from `last_set_ids`; otherwise, last_set_id = elem_id
469-
5. Consume one value from the visible values list
476+
The first item in `items` is a sentinel. For each item record:
477+
478+
1. **Skip first iteration** (sentinel has no visible item to decode)
479+
2. **For non-first iterations**, consume the visible item:
480+
- Consume one position ID from `list_item_ids`
481+
- If `pos_id_eq_elem_id` is false, consume from `elem_ids`; otherwise elem_id = position_id.idlp()
482+
- If `elem_id_eq_last_set_id` is false, consume from `last_set_ids`; otherwise last_set_id = elem_id
483+
- Consume one value from the visible values list
484+
- Push the visible item
485+
3. **After the visible item**, consume `invisible_list_item` invisible positions:
486+
- For each: consume one position ID from `list_item_ids`, push as invisible item (no value)
487+
488+
**Key insight**: During encoding, when an invisible item is encountered, it increments the
489+
**previous** visible item's `invisible_list_item` counter (see line 1509). This means each
490+
record's `invisible_list_item` represents invisible items that **follow** the visible item.
470491

471492
**Source**: `crates/loro-internal/src/state/movable_list_state.rs:1392-1637`
472493

@@ -543,6 +564,7 @@ function decodeContainerState(bytes) {
543564
case 2: return { type: 'text', state: decodeRichtextState(stateBytes) };
544565
case 3: return { type: 'tree', state: decodeTreeState(stateBytes) };
545566
case 4: return { type: 'movable_list', state: decodeMovableListState(stateBytes) };
567+
case 5: return { type: 'counter', state: decodeCounterState(stateBytes) }; // counter feature
546568
default: throw new Error(`Unknown container type: ${containerType}`);
547569
}
548570
}

0 commit comments

Comments
 (0)