Skip to content

Performance regression: lite-scheduler I/O benchmarks (~20-27% slower) #8030

Description

@westonpace

The lance-bench run for Lance b29e602fd781e2076e86d87b5d425bb4991abbb0 detected two high-confidence I/O regressions in the last-4-results statistical analysis:

  • from_elem/random_read,parallel=16,item_size=4096,use_lite_scheduler=Some(true),noisy=false: p=0.003845, +26.85% slower
  • from_elem/full_read,parallel=16,read_size=4096,use_lite_scheduler=Some(true): p=0.008283, +20.42% slower

Run: https://github.com/lancedb/lance-bench/actions/runs/30317331277

Both signals specifically use the lite scheduler at parallelism 16, suggesting a shared scheduler/concurrency regression. The benchmark compares the four most recent measurements with all older measurements. Recent Lance history immediately before the tested SHA includes b29e602 (bound SlotBackoff retry sleeps), 88d4b14 (CPU/IO env validation), and a15c0bc / c9178fe (quick_cache changes), but the benchmark data does not establish causality; please reproduce/bisect the lite-scheduler cases across the recent four-result window.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions