Skip to content

Commit ba4800d

Browse files
Dandandanclaude
andcommitted
Optimize insert probing with early EMPTY check
In find_or_find_insert_index_inner, when the first empty-or-deleted slot found is EMPTY (not a tombstone), return immediately without the expensive match_empty() SIMD group scan. This is the common case when there are no tombstones in the table. An EMPTY byte at the computed index also guarantees the index is valid (not a small-table wrapping false positive), so the fix_insert_index check is skipped too. Benchmarks (aarch64, 10 runs, median): insert_foldhash_serial: 11,734 -> 11,410 ns (-2.8%) rehash_in_place: 220,745 -> 189,163 ns (-14.3%) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 7e10037 commit ba4800d

1 file changed

Lines changed: 18 additions & 0 deletions

File tree

src/raw.rs

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1832,6 +1832,24 @@ impl RawTableInner {
18321832
// insertion slot from the group if we don't have one yet.
18331833
if likely(insert_index.is_none()) {
18341834
insert_index = self.find_insert_index_in_group(&group, &probe_seq);
1835+
1836+
// When the found slot is EMPTY (not DELETED), we can stop probing
1837+
// immediately without computing the more expensive match_empty() on
1838+
// the whole group. This is the common case when there are no
1839+
// tombstones in the table.
1840+
//
1841+
// An EMPTY slot in this group guarantees the key we're looking for
1842+
// can't be in a subsequent group. Additionally, a true EMPTY byte
1843+
// at the computed index means the index is valid (not a small-table
1844+
// false positive from wrapping), so fix_insert_index is unnecessary.
1845+
if let Some(idx) = insert_index {
1846+
// SAFETY: The index was found by `find_insert_index_in_group` which
1847+
// guarantees it is in the range `0..=self.bucket_mask`, so calling
1848+
// `self.ctrl(idx)` is safe.
1849+
if likely(unsafe { *self.ctrl(idx) } == Tag::EMPTY) {
1850+
return Err(idx);
1851+
}
1852+
}
18351853
}
18361854

18371855
if let Some(insert_index) = insert_index {

0 commit comments

Comments
 (0)