Commit 1e93ddc
committed
streaming_chunker: don't emit boundary on internal-level chunks with <2 entries
The Python CI hit a segfault (release) / u8 overflow panic (debug) in
basic_usage.py at the file-backed `insert` step. Root cause: Python's
`TreeConfig.__init__` has `pattern=0` as its default, and the example
also sets `min_chunk_size=1`. With pattern=0 the splitter's
`(hash & pattern) == pattern` check is true on every item, so a
boundary fires on every append. The chunker's `handle_boundary` then
recursively created a new parent chunker one level up, whose splitter
saw the just-emitted `(firstKey, hash)` entry, fired a boundary, and
cascaded - climbing the `u8` level counter until it overflowed.
This is Dolt's "constraint (3): internal nodes must contain at least
2 key-value pairs" rule from `chunker.go::append`, which we didn't
port. Adding it: at level > 0, defer boundary emission until the
in-progress chunk has at least 2 entries. Leaf-level chunks (level 0)
are still allowed to be single-entry so we can store a 1-key dataset.
The old in-place `Balanced::balance` didn't have this bug because it
emitted upward only when the chunker reported >1 chunks (its
`chunks.len() <= 1` early-return). The streaming chunker emits at
every boundary, so we need the explicit constraint here.
Regression test in `streaming_chunker::tests` reproduces the
degenerate config and asserts the level stays bounded.
Verified by re-running basic_usage.py against a debug + release build,
both succeed end-to-end. Rust suite: 160 lib tests pass, 0 failed.1 parent 66cf674 commit 1e93ddc
1 file changed
Lines changed: 48 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
352 | 352 | | |
353 | 353 | | |
354 | 354 | | |
355 | | - | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
356 | 367 | | |
357 | 368 | | |
358 | 369 | | |
| |||
1144 | 1155 | | |
1145 | 1156 | | |
1146 | 1157 | | |
| 1158 | + | |
| 1159 | + | |
| 1160 | + | |
| 1161 | + | |
| 1162 | + | |
| 1163 | + | |
| 1164 | + | |
| 1165 | + | |
| 1166 | + | |
| 1167 | + | |
| 1168 | + | |
| 1169 | + | |
| 1170 | + | |
| 1171 | + | |
| 1172 | + | |
| 1173 | + | |
| 1174 | + | |
| 1175 | + | |
| 1176 | + | |
| 1177 | + | |
| 1178 | + | |
| 1179 | + | |
| 1180 | + | |
| 1181 | + | |
| 1182 | + | |
| 1183 | + | |
| 1184 | + | |
| 1185 | + | |
| 1186 | + | |
| 1187 | + | |
| 1188 | + | |
| 1189 | + | |
| 1190 | + | |
| 1191 | + | |
| 1192 | + | |
| 1193 | + | |
1147 | 1194 | | |
0 commit comments