Skip to content

Feat/query memory perf#11

Merged
St4NNi merged 28 commits into
mainfrom
feat/query_memory_perf
May 28, 2026
Merged

Feat/query memory perf#11
St4NNi merged 28 commits into
mainfrom
feat/query_memory_perf

Conversation

@St4NNi

@St4NNi St4NNi commented May 28, 2026

Copy link
Copy Markdown
Owner

This PR polishes the tool and moves it to a more production ready state.

Changes

  • Reworks query/dist execution for lower memory use with chunked matching, deduplicated query hashes, cutoff filtering, bias-weighted containment, and uniform-hash e-values.
  • Updates bias-aware sketching and CLI around target/max/unseen fscale calibration, default fscale behavior, and clearer bias-mode reporting/docs.
  • Hardens JAM reader/writer/sketch validation against malformed metadata, overflows, invalid inputs, and FASTQ/file-list edge cases.
  • Adds K21 bias/hash benchmarking pipeline tooling, SLURM scripts, benchmark coverage, dependency updates, and CI/Docker maintenance.

@St4NNi St4NNi merged commit 774e3b1 into main May 28, 2026
2 checks passed
@codecov

codecov Bot commented May 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.76616% with 573 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.02%. Comparing base (7253622) to head (b1bae03).
⚠️ Report is 29 commits behind head on main.

Files with missing lines Patch % Lines
src/cli/handlers.rs 14.42% 178 Missing ⚠️
src/reader.rs 66.50% 131 Missing and 7 partials ⚠️
src/bias.rs 84.63% 85 Missing and 19 partials ⚠️
src/writer.rs 56.37% 48 Missing and 17 partials ⚠️
src/query.rs 88.79% 43 Missing and 11 partials ⚠️
src/lib.rs 0.00% 15 Missing ⚠️
src/sketch.rs 88.03% 11 Missing and 3 partials ⚠️
src/io.rs 87.17% 4 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             main      #11     +/-   ##
=========================================
  Coverage   72.02%   72.02%             
=========================================
  Files          11       11             
  Lines        5655     7082   +1427     
  Branches     5655     7082   +1427     
=========================================
+ Hits         4073     5101   +1028     
- Misses       1454     1814    +360     
- Partials      128      167     +39     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant