Skip to content

Index optimized for small data#840

Open
yihozhang wants to merge 3 commits intoyihozhang-decompfrom
yihozhang-decomp-small-index
Open

Index optimized for small data#840
yihozhang wants to merge 3 commits intoyihozhang-decompfrom
yihozhang-decomp-small-index

Conversation

@yihozhang
Copy link
Copy Markdown
Collaborator

4/5 of the indices built for some large benchmarks only have one element. It is expensive to build a ColumnIndex for them.

This PR is currently bottlenecked on scanning and materializing the rows corresponding to the Subset.

@yihozhang yihozhang requested a review from a team as a code owner April 9, 2026 08:52
@yihozhang yihozhang requested review from saulshanabrook and removed request for a team April 9, 2026 08:52
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 9, 2026

Codecov Report

❌ Patch coverage is 94.54545% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.94%. Comparing base (0a5967e) to head (948e30c).
⚠️ Report is 2 commits behind head on yihozhang-decomp.

Files with missing lines Patch % Lines
core-relations/src/row_buffer/mod.rs 90.59% 11 Missing ⚠️
core-relations/src/free_join/execute.rs 99.00% 1 Missing ⚠️
Additional details and impacted files
@@                 Coverage Diff                  @@
##           yihozhang-decomp     #840      +/-   ##
====================================================
+ Coverage             86.90%   86.94%   +0.04%     
====================================================
  Files                    85       85              
  Lines                 24326    24475     +149     
====================================================
+ Hits                  21140    21280     +140     
- Misses                 3186     3195       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Apr 9, 2026

Merging this PR will improve performance by 24.19%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 9 improved benchmarks
✅ 35 untouched benchmarks
⏩ 190 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime tests[eggcc-extraction] 1.8 s 1.7 s +6.04%
WallTime tests[hardboiled_conv1d_32] 1.7 s 1.4 s +16.98%
WallTime tests[hardboiled_conv1d_128] 6.7 s 5.5 s +21.38%
WallTime tests[herbie] 272.5 ms 254.9 ms +6.89%
Simulation tests[math-microbenchmark] 8.9 s 8.5 s +5.1%
WallTime tests[proof_testing_typecheck] 311.2 ms 293.6 ms +5.99%
WallTime tests[python_array_optimize] 6.9 s 6.5 s +5.67%
Simulation tests[hardboiled_conv1d_32] 1.8 s 1.7 s +8.02%
Simulation tests[hardboiled_conv1d_128] 8.9 s 7.2 s +24.19%

Comparing yihozhang-decomp-small-index (948e30c) with yihozhang-decomp (22d06c1)

Open in CodSpeed

Footnotes

  1. 190 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@yihozhang yihozhang force-pushed the yihozhang-decomp-small-index branch from 8da8493 to d6daffc Compare April 12, 2026 08:28
@yihozhang yihozhang requested review from ezrosent and removed request for saulshanabrook April 18, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants