Replace BTreeMap usage with hashbrown hashmap#221
Merged
Conversation
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
bkchr
reviewed
May 22, 2025
|
|
||
| #[cfg(not(feature = "std"))] | ||
| use alloc::collections::btree_map::{BTreeMap as Map, Entry}; | ||
| use hashbrown::{hash_map::Entry, HashMap as Map}; |
Member
There was a problem hiding this comment.
We can just use it on std and no_std. The std hashmap is actually hashbrown any way.
bkchr
approved these changes
May 22, 2025
skunert
approved these changes
May 22, 2025
lexnv
approved these changes
May 22, 2025
AndreiEres
approved these changes
May 22, 2025
Merged
github-merge-queue bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
May 23, 2025
Discovered while profiling #6131 (comment) with the benchmark #8069 that when running in validation a big chunk of the time is spent inserting and retrieving data from the BTreeMap/BTreeSet. By switching to hashbrown HashMap/HashSet in validation TrieCache and TrieRecorder and the memory-db paritytech/trie#221 read costs improve with around ~40% and write with about ~20% --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
github-merge-queue bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
May 27, 2025
Bump memory-db to pick up #8606 and paritytech/trie#221 Additionally, polkavm needs to be bumped to get rid of to get rid of https://github.com/paritytech/polkadot-sdk/actions/runs/15180236627/job/42688141374#step:5:1869 --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
pgherveou
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Jun 11, 2025
Discovered while profiling #6131 (comment) with the benchmark #8069 that when running in validation a big chunk of the time is spent inserting and retrieving data from the BTreeMap/BTreeSet. By switching to hashbrown HashMap/HashSet in validation TrieCache and TrieRecorder and the memory-db paritytech/trie#221 read costs improve with around ~40% and write with about ~20% --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
pgherveou
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Jun 11, 2025
Bump memory-db to pick up #8606 and paritytech/trie#221 Additionally, polkavm needs to be bumped to get rid of to get rid of https://github.com/paritytech/polkadot-sdk/actions/runs/15180236627/job/42688141374#step:5:1869 --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
github-merge-queue bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Jul 11, 2025
…lidation context (#9127) #8606 paritytech/trie#221 replaced the usage of BTreeMap with HashMaps in validation context. The keys are already derived with a cryptographic hash function from user data, so users should not be able to manipulate it. To be on safe side this PR also modifies the TrieCache, TrieRecorder and MemoryDB to use a hasher that on top of the default generated randomness also adds randomness generated from the hash of the relaychain and that of the parachain blocks, which is not something users can control or guess ahead of time. --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
paritytech-release-backport-bot bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Jul 11, 2025
…lidation context (#9127) #8606 paritytech/trie#221 replaced the usage of BTreeMap with HashMaps in validation context. The keys are already derived with a cryptographic hash function from user data, so users should not be able to manipulate it. To be on safe side this PR also modifies the TrieCache, TrieRecorder and MemoryDB to use a hasher that on top of the default generated randomness also adds randomness generated from the hash of the relaychain and that of the parachain blocks, which is not something users can control or guess ahead of time. --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> (cherry picked from commit 7058819)
paritytech-release-backport-bot bot
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Jul 17, 2025
…lidation context (#9127) #8606 paritytech/trie#221 replaced the usage of BTreeMap with HashMaps in validation context. The keys are already derived with a cryptographic hash function from user data, so users should not be able to manipulate it. To be on safe side this PR also modifies the TrieCache, TrieRecorder and MemoryDB to use a hasher that on top of the default generated randomness also adds randomness generated from the hash of the relaychain and that of the parachain blocks, which is not something users can control or guess ahead of time. --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com> (cherry picked from commit 7058819)
alvicsam
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Oct 17, 2025
Discovered while profiling #6131 (comment) with the benchmark #8069 that when running in validation a big chunk of the time is spent inserting and retrieving data from the BTreeMap/BTreeSet. By switching to hashbrown HashMap/HashSet in validation TrieCache and TrieRecorder and the memory-db paritytech/trie#221 read costs improve with around ~40% and write with about ~20% --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
alvicsam
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Oct 17, 2025
Bump memory-db to pick up #8606 and paritytech/trie#221 Additionally, polkavm needs to be bumped to get rid of to get rid of https://github.com/paritytech/polkadot-sdk/actions/runs/15180236627/job/42688141374#step:5:1869 --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
alvicsam
pushed a commit
to paritytech/polkadot-sdk
that referenced
this pull request
Oct 17, 2025
…lidation context (#9127) #8606 paritytech/trie#221 replaced the usage of BTreeMap with HashMaps in validation context. The keys are already derived with a cryptographic hash function from user data, so users should not be able to manipulate it. To be on safe side this PR also modifies the TrieCache, TrieRecorder and MemoryDB to use a hasher that on top of the default generated randomness also adds randomness generated from the hash of the relaychain and that of the parachain blocks, which is not something users can control or guess ahead of time. --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: Bastian Köcher <git@kchr.de> Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Discovered while profiling paritytech/polkadot-sdk#6131 (comment) with the benchmark paritytech/polkadot-sdk#8069 that when running in validation a big chunk of the time is spent inserting and retrieving data from the BTreeMap/BTreeSet.
This PR together with paritytech/polkadot-sdk#8606 improve read costs with around ~40% and write costs with about ~20%