Count content IDs with direct indexing

AI-assisted change (OpenAI Codex).

Author: Wilfred

PERF_RESEARCH_LOG.md

@@ -1396,6 +1396,28 @@ side-by-side, inline, and JSON modes on all 107 available sample pairs.
 `cargo test` passes (164 passed, one ignored), and the changed file passes
 `rustfmt --check`.
 
+### exp46: count dense content IDs in a vector — KEPT
+
+The two uniqueness passes counted `ContentId`s in hash maps. Like syntax IDs,
+content IDs are dense positive integers; after both trees receive IDs, the
+number of possible IDs is already known from the content-key table. The counts
+now use direct vector indexing, and the allocation is cleared and reused for
+the RHS. This removes hashing from both the counting and uniqueness-setting
+walks; exp38 had shown that eliminating a walk while retaining those hashes was
+flat.
+
+Ten-run `perf stat` means were:
+
+| pair | hashed counts | dense counts | change |
+| --- | ---: | ---: | ---: |
+| `slow` | 514,819,594 | 514,223,778 | **-0.116%** |
+| `typing` | 2,754,599,330 | 2,743,556,655 | **-0.401%** |
+
+Peak RSS was flat in single-run checks. Output is byte-identical in
+side-by-side, inline, and JSON modes on all 107 available sample pairs.
+`cargo test` passes (164 passed, one ignored), and the changed file passes
+`rustfmt --check`.
+
 ## Where this leaves things
 
 | workload | earlier reference | now | change |
@@ -1404,8 +1426,8 @@ side-by-side, inline, and JSON modes on all 107 available sample pairs.
 | trivial Rust diff, master through query work | 444,668,795 | ~206,000,000 | **-54%** |
 | `huge_cpp`, before exp17 through exp22 | 14,858,905,910 | 9,285,036,692 | **-37.5%** |
 | `huge_cpp` peak RSS, exp21 through exp22 | 696 MB | 491 MB | **-29%** |
-| focused `typing`, exp22 through exp45 | 3,115,140,742 | 2,754,519,861 | **-11.58%** |
-| focused `slow`, exp22 through exp45 | 1,881,539,982 | 514,830,042 | **-72.64%** |
+| focused `typing`, exp22 through exp46 | 3,115,140,742 | 2,743,556,655 | **-11.93%** |
+| focused `slow`, exp22 through exp46 | 1,881,539,982 | 514,223,778 | **-72.67%** |
 
 The large-input pass found a different class of wins from the original suite:
 quadratic display loops (exp13-17), redundant offset-to-line searches (exp19),
@@ -1414,7 +1436,7 @@ The post-exp22 `huge_cpp` profile is now led by line splitting, Imara histogram
 diff construction, allocator traffic, and changed-region line conversion; the
 previous hunk-end and opposite-line hash hotspots are gone.
 
-The focused exp23-45 pass first found small, composable wins in slider range
+The focused exp23-46 pass first found small, composable wins in slider range
 collection and parent-stack dispatch, then a much larger win by decomposing
 oversized graphs at similar sibling lists. It also ruled out three tempting
 graph directions on the exact target inputs: regenerating cached neighbours,

plan.md

@@ -241,10 +241,10 @@ new incremental experiment, use the latest accepted branch tip as the control.
 
 Fresh five-run `perf stat` means at the start of the focused pass:
 
-| pair | post-exp22 instructions | latest accepted after exp45 | cumulative change |
+| pair | post-exp22 instructions | latest accepted after exp46 | cumulative change |
 | --- | ---: | ---: | ---: |
-| `typing` | 3,115,140,742 | 2,754,519,861 | **-11.58%** |
-| `slow` | 1,881,539,982 | 514,830,042 | **-72.64%** |
+| `typing` | 3,115,140,742 | 2,743,556,655 | **-11.93%** |
+| `slow` | 1,881,539,982 | 514,223,778 | **-72.67%** |
 
 The original focused callgrind files were `/tmp/cg-focused-typing.out` and
 `/tmp/cg-focused-slow.out`; sampled cycle profiles were
@@ -362,6 +362,7 @@ than mixing counts across environments.
 | 43 | reserve matched-position capacity from syntax-node count | flat on both focused pairs | rejected |
 | 44 | reuse one slider-region scratch vector across both passes | flat on both focused pairs | rejected |
 | 45 | replace hashed `ChangeMap` with dense syntax-ID indexing | -1.20% `typing`, -0.27% `slow`; `typing` RSS -4.5% | kept |
+| 46 | replace hashed content-ID counts with a dense vector | -0.40% `typing`, -0.12% `slow` | kept |
 
 Every completed experiment is committed and pushed. `results/` holds labelled
 callgrind tables through `exp13-linear-visible-width`; experiments that only
@@ -370,8 +371,8 @@ affect inline, JSON, or synthetic/very-large shapes use repeated targeted
 `huge_cpp` pair from 14.86B to 9.29B instructions (**-37.5%**), and exp22 also
 reduced its peak RSS from 696 MB to 491 MB. If the compiler, dependencies,
 profiler, or machine changed, record a fresh control binary before comparing.
-The focused exp23-45 pass reduced `typing` from 3.115B to 2.755B (**-11.58%**)
-and `slow` from 1.882B to 0.515B (**-72.64%**). Exp35's similar-list pairing and
+The focused exp23-46 pass reduced `typing` from 3.115B to 2.744B (**-11.93%**)
+and `slow` from 1.882B to 0.514B (**-72.67%**). Exp35's similar-list pairing and
 exp39's lower decomposition gate are responsible for most of that improvement.
 
 ## Research synthesis: where larger wins can come from
@@ -625,9 +626,10 @@ percentages above predate exp25-31.
    cache and packed `VertexKey`. Look for repeated state extraction, parent
    stack allocation, or matching in existing edges. Exp28 and exp31 show exact
    parent-stack reductions pay; exp10 shows merely decomposing candidate
-   construction does not. Exp45 also shows that dense sequential identifiers
-   should use direct indexing rather than hashing; audit other hot maps for the
-   same invariant before changing their representation.
+   construction does not. Exp45-46 show that dense sequential identifiers
+   should use direct indexing rather than hashing. The remaining content-key
+   table is keyed by structural content and cannot use this technique; audit
+   other hot maps only when they have the same dense invariant.
 8. **Target the remaining mixed-pipeline work on `typing`.** Exp27 proved
    capture-bucket lookup itself is negligible, while exp41 removed one
    allocation from `change_positions_`, and exp42 removed the per-node result
@@ -689,7 +691,7 @@ percentages above predate exp25-31.
    rather than comparing across machines or toolchains.
 5. Choose one hypothesis from the prioritised backlog, state whether it is in
    the exact-output or diff-quality lane, change one thing, measure both pairs,
-   and immediately append exp46 (then exp47, etc.) to
+   and immediately append exp47 (then exp48, etc.) to
    `PERF_RESEARCH_LOG.md` and the state table here.
 6. Fully revert rejected source changes with `apply_patch`, but commit and push
    their log entries. For a kept change, run the wider output oracle and full

src/parse/syntax.rs

@@ -440,8 +440,12 @@ fn init_info<'a>(lhs_roots: &[&'a Syntax<'a>], rhs_roots: &[&'a Syntax<'a>]) {
     set_content_id(lhs_roots, &mut existing);
     set_content_id(rhs_roots, &mut existing);
 
-    set_content_is_unique(lhs_roots);
-    set_content_is_unique(rhs_roots);
+    // Content IDs are dense and start at one. Keep slot zero unused so an ID
+    // can index this vector directly.
+    let mut content_counts = vec![0; existing.len() + 1];
+    set_content_is_unique(lhs_roots, &mut content_counts);
+    content_counts.fill(0);
+    set_content_is_unique(rhs_roots, &mut content_counts);
 }
 
 type ContentKey = (Option<String>, Option<String>, Vec<u32>, bool, bool);
@@ -532,32 +536,30 @@ fn set_identity_and_ancestry<'a>(
 }
 
 /// Assumes that `set_content_id` has already run.
-fn find_nodes_with_unique_content(nodes: &[&Syntax], counts: &mut DftHashMap<ContentId, usize>) {
+fn find_nodes_with_unique_content(nodes: &[&Syntax], counts: &mut [usize]) {
     for node in nodes {
-        *counts.entry(node.content_id()).or_insert(0) += 1;
+        counts[node.content_id() as usize] += 1;
         if let List { children, .. } = node {
             find_nodes_with_unique_content(children, counts);
         }
     }
 }
 
-fn set_content_is_unique_from_counts(nodes: &[&Syntax], counts: &DftHashMap<ContentId, usize>) {
+fn set_content_is_unique_from_counts(nodes: &[&Syntax], counts: &[usize]) {
     for node in nodes {
-        let count = counts
-            .get(&node.content_id())
-            .expect("Count should be present");
-        node.info().content_is_unique_to_side.set(*count == 1);
+        node.info()
+            .content_is_unique_to_side
+            .set(counts[node.content_id() as usize] == 1);
 
         if let List { children, .. } = node {
             set_content_is_unique_from_counts(children, counts);
         }
     }
 }
 
-fn set_content_is_unique(nodes: &[&Syntax]) {
-    let mut counts = DftHashMap::default();
-    find_nodes_with_unique_content(nodes, &mut counts);
-    set_content_is_unique_from_counts(nodes, &counts);
+fn set_content_is_unique(nodes: &[&Syntax], counts: &mut [usize]) {
+    find_nodes_with_unique_content(nodes, counts);
+    set_content_is_unique_from_counts(nodes, counts);
 }
 
 /// Initialise the sibling links and the preceding node used by graph search in