Skip to content

Commit dfb0bfb

Browse files
committed
docs: add an update_deletion_vectors data-shape example (#3067)
Add a doc-comment example to \Transaction::update_deletion_vectors\ showing the expected input shapes from scan metadata and the DV map, how Kernel stages the row with temporary columns, and what Add/Remove actions are produced on commit. Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com>
1 parent 57ad014 commit dfb0bfb

1 file changed

Lines changed: 37 additions & 0 deletions

File tree

kernel/src/transaction/update.rs

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -340,6 +340,43 @@ impl Transaction {
340340
///
341341
/// On commit, each matched file's add action carries `stats.tightBounds: false`.
342342
///
343+
/// # Data Shape & Staging
344+
///
345+
/// Given scan metadata input and a map of new deletion vector descriptors:
346+
///
347+
/// ```text
348+
/// scan metadata input:
349+
///
350+
/// path size stats deletionVector
351+
/// part-1.parquet 100 {"numRecords":10,"tightBounds":true} old_dv
352+
/// part-2.parquet 200 {"numRecords":20,"tightBounds":true} null
353+
///
354+
/// DV map:
355+
///
356+
/// {
357+
/// "part-1.parquet": new_dv
358+
/// }
359+
/// ```
360+
///
361+
/// Kernel matches the map by `path` and stages the selected scan row with two temporary columns:
362+
///
363+
/// ```text
364+
/// path ... deletionVector newDeletionVector newStats
365+
/// part-1.parquet ... old_dv new_dv {"numRecords":10,"tightBounds":false}
366+
/// ```
367+
///
368+
/// On commit, this produces:
369+
///
370+
/// ```text
371+
/// Remove(path="part-1.parquet", deletionVector=old_dv)
372+
/// Add(
373+
/// path="part-1.parquet",
374+
/// deletionVector=new_dv,
375+
/// stats={"numRecords":10,"tightBounds":false},
376+
/// ...preserved file metadata
377+
/// )
378+
/// ```
379+
///
343380
/// # Arguments
344381
///
345382
/// * `new_dv_descriptors` - A map from data file path (as provided in scan operations) to the

0 commit comments

Comments
 (0)