[0.9.10] Final release before 1.0

After a long time, I am proud to present the first stable release of Jam, my new genomic minhashing tool. Jam emphasizes performance and accuracy, and it aims to make searching for "small" sequences, such as plasmids, phages, and mobile elements, more efficient in "large" genomic sets, such as metagenomes.

What's Changed

• Added jamhash dependency, benchmark jamhash improve jamhash performance, in-depth analysis compared to murmur3
• Removed LMDB support, added new custom storage format
• Removed GC / Length tracking, added full bias table support using Count-min-sketches
• Added fast pre-filter using state-of-the-art BinaryFuseFilters
• Heavily optimized performance, reduced IO and memory load.
• Added mmap support for .jam files
• Added .bias files for pre-computed bias tables
• Added shannon-entropy low complexity filtering

Full Changelog: v0.1.0-beta.2...v0.9.10

v0.1.0-beta.2

First pre-release v0.1.0-beta.2

Same as beta.1, fixes an issue with the binary package on crates.io

After a far too long hiatus, it is time to announce the first official pre-release of jam-rs. This version includes all the basic features for sketching and comparing multiple signatures.

You can install it right now via:

> cargo install jam-rs@0.1.0-beta.2

Hashing algorithms

The sketching part includes several configurable algorithms:

• (legacy) Murmurhash3, which produces the same results as other genomic minhashing tools
• xxhash3, a newer algorithm with similar characteristics to murmur3
• Ahash-fallback, an optimised adaptation of the fallback algorithm for small inputs from the Ahash crate. This algorithm is only usable for kmer < 32, but preliminary statistical analysis shows that it produces hashes that are more evenly distributed, with a 3-10x overall speedup compared to xxhash or murmur.

You can choose between two output formats after sketching: sourmash-compatible JSON or a more compact binary representation with additional meta-information. The sourmash-compatible output can be used directly with the regular sourmash pipeline, but note that comparing sketches with non-murmur hashing algorithms will only work if both sides use the same algorithm.

Scaling factors

This tool includes both of the scaling methods from mash and sourmash, plus a combination of both, where a fixed fraction of the hash space is stored, with a fixed lower and/or upper limit per sequence. This ensures that all sequences are covered, with the disadvantage of introducing a bias towards smaller (or larger) sequences. This makes it necessary to perform more statistics too make sure that hits do not occur by chance which will follow soon.

What's Changed

• [feat] Feat/hash algs by @St4NNi in #1
• [tests] Feat/refactor tests by @St4NNi in #2
• [0.1.0-beta.1] Feat/compare by @St4NNi in #3

New Contributors

• @St4NNi made their first contribution in #1

Full Changelog: https://github.com/St4NNi/jam-rs/commits/v0.1.0-beta.1

Disclaimer

This is still a pre-release, check it out but take the results with a grain of salt. Additional tests and verifications are in progress and will be released before the final first release.

v0.1.0-beta.1

First pre-release v0.1.0-beta.1

After a far too long hiatus, it is time to announce the first official pre-release of jam-rs. This version includes all the basic features for sketching and comparing multiple signatures.

Hashing algorithms

The sketching part includes several configurable algorithms:

• (legacy) Murmurhash3, which produces the same results as other genomic minhashing tools
• xxhash3, a newer algorithm with similar characteristics to murmur3
• Ahash-fallback, an optimised adaptation of the fallback algorithm for small inputs from the Ahash crate. This algorithm is only usable for kmer < 32, but preliminary statistical analysis shows that it produces hashes that are more evenly distributed, with a 3-10x overall speedup compared to xxhash or murmur.

You can choose between two output formats after sketching: sourmash-compatible JSON or a more compact binary representation with additional meta-information. The sourmash-compatible output can be used directly with the regular sourmash pipeline, but note that comparing sketches with non-murmur hashing algorithms will only work if both sides use the same algorithm.

Scaling factors

This tool includes both of the scaling methods from mash and sourmash, plus a combination of both, where a fixed fraction of the hash space is stored, with a fixed lower and/or upper limit per contig. This ensures that all contigs are covered, with the disadvantage of introducing a bias towards smaller (or larger) sequences. This makes it necessary to perform more statistics too make sure that hits do not occur by chance which will follow soon.

What's Changed

• [feat] Feat/hash algs by @St4NNi in #1
• [tests] Feat/refactor tests by @St4NNi in #2
• [0.1.0-beta.1] Feat/compare by @St4NNi in #3

New Contributors

• @St4NNi made their first contribution in #1

Full Changelog: https://github.com/St4NNi/jam-rs/commits/v0.1.0-beta.1

Disclaimer

This is still a pre-release, check it out but take the results with a grain of salt. Additional tests and verifications are in progress and will be released before the final first release.