Skip to content

Commit 128b466

Browse files
committed
docs: fix value tag 7 description and serde_columnar format
1. Value tag 7 (ContainerType): clarify it's a container reference (index into ContainerArena), not a "type marker". The actual container type is stored in the ContainerArena entry. Source: value.rs:357,422 shows ContainerIdx(usize) encoding. 2. serde_columnar outer format: add missing column-count prefix and fix "LEB128" to "postcard varint". Without the count, decoders would misread the first varint and get out of sync.
1 parent 82415df commit 128b466

File tree

1 file changed

+10
-10
lines changed

1 file changed

+10
-10
lines changed

docs/encoding.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -689,7 +689,7 @@ Loro uses a tagged value encoding system where each value is prefixed with a typ
689689
| 4 | F64 | 64-bit floating point (big-endian) |
690690
| 5 | Str | String value (LEB128 len + UTF-8 bytes) |
691691
| 6 | Binary | Binary data (LEB128 len + bytes) |
692-
| 7 | ContainerType | Container type marker (LEB128 index) |
692+
| 7 | ContainerType | Container reference (LEB128 index into change block ContainerArena `cids`) |
693693
| 8 | DeleteOnce | Single deletion marker |
694694
| 9 | DeleteSeq | Sequence deletion |
695695
| 10 | DeltaInt | Delta-encoded i32 (LEB128 signed) |
@@ -1146,24 +1146,24 @@ Example: Encoding [1, 2, 3, 4, 5, 6]
11461146

11471147
### Columnar Encoding (serde_columnar)
11481148

1149-
The `serde_columnar` library provides columnar storage with per-column compression:
1149+
The `serde_columnar` library provides columnar storage with per-column compression. The outer wire
1150+
format is a postcard sequence of column payloads:
11501151

11511152
```
11521153
┌──────────────────────────────────────────────────────────────────┐
1153-
│ serde_columnar Format
1154+
│ serde_columnar Outer Format
11541155
├───────────────┬──────────────────────────────────────────────────┤
1156+
│ postcard varint│ Number of columns (fields) │
11551157
│ For each column: │
1156-
LEB128 │ Column data length
1158+
postcard varint│ Column data length in bytes
11571159
│ bytes │ Column data (strategy-dependent) │
1158-
│ │ │
1159-
│ Strategies: │ │
1160-
│ Rle │ Run-length encoded values │
1161-
│ DeltaRle │ Delta + RLE │
1162-
│ BoolRle │ Boolean RLE │
1163-
│ Raw │ No compression │
1160+
│ │ Row count inferred from column payload │
11641161
└───────────────┴──────────────────────────────────────────────────┘
11651162
```
11661163

1164+
Common strategies used by Loro include `BoolRle`, `Rle`, `DeltaRle`, and `DeltaOfDelta` (see below
1165+
and [External Format Specifications](#external-format-specifications) for details).
1166+
11671167
### LZ4 Compression
11681168

11691169
Block-level compression for SSTable blocks:

0 commit comments

Comments
 (0)