Commit 2832ac3
Fix empty-index BLAS search results (#5528)
Summary:
- initialize result handlers when an exhaustive search has no database vectors
- process empty results in the configured query block size, preserving bounded temporary memory for large query batches
- cover L2 and inner-product search across top-1, heap, and reservoir result handlers
## Problem
Searching an empty `IndexFlat` produced correct sentinel values for small query batches, but could leave the caller's distance and label buffers untouched once `nq * d` selected the BLAS path. The BLAS helpers return early for `ny == 0` before the result handler's `begin_multiple` / `end_multiple` lifecycle initializes and finalizes the output.
This change handles an empty database before choosing the sequential or BLAS implementation. It uses the existing result-handler lifecycle so each handler supplies its own correct neutral distance and `-1` label semantics.
Fixes #3830.
## Validation
- `python -m unittest test_index.TestIndexFlat_ARM_NEON.test_empty_index_with_blas`
- `python -m unittest test_index` (49 tests passed)
- `faiss_test --gtest_brief=1` (290 passed, 1 platform-specific skip)
- `clang-format 21 --dry-run --Werror faiss/utils/distances.cpp`
## Verification of the regression test
We confirmed `test_empty_index_with_blas` is a true regression test by running it with and without the `utils/distances.cpp` change:
| State | Result |
|-------|--------|
| Fix applied | 1 passed, 0 failed |
| `utils/distances.cpp` reverted to parent, test retained | 6 failed |
The 6 failures are every subtest (2 metrics x 3 values of `k`), covering the `Top1`, `Heap`, and `Reservoir` result handlers. The failure output reports `ACTUAL: array([[42., 42., ...` against `DESIRED: array(-3.402823e+38)`. The test pre-fills `D` with `42` and `I` with `99`, so this confirms the output buffers are never written without the fix, rather than written with incorrect values. The passing run is clean under ASan and UBSan.
## Note for issue #3830
The repro in the issue (`d=1`, `nq=20`) no longer reproduces on `main`. When the issue was filed, `distance_compute_blas_threshold` defaulted to `20` and the branch condition was `nx < threshold`, which put `nq=19` on the sequential path (correct sentinels) and `nq=20` on the BLAS path (uninitialized output). The threshold is now dimension-aware (`nx * d < distance_compute_blas_threshold`, default `128000`), so `d=1` requires `nq >= 128000` to select BLAS. The defect was relocated, not fixed. A repro that still holds today is:
```
faiss.IndexFlatL2(128).search(np.random.random((1000, 128)).astype('float32'), 1) # nx * d == 128000
```
This change removes the defect at its source, for all `nx`, all `d`, both metrics, and all three result handlers. The `k > ntotal` case with `ntotal > 0` was never affected: `begin_multiple` already initializes all `k` slots, so the unfilled tail is correctly padded.
Pull Request resolved: #5528
Reviewed By: mnorris11, trang-nm-nguyen
Differential Revision: D116650117
Pulled By: alibeklfc
fbshipit-source-id: b40bdf7ade7640b2e433765b3877e2cea3546cf21 parent e135a1a commit 2832ac3
2 files changed
Lines changed: 33 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
563 | 563 | | |
564 | 564 | | |
565 | 565 | | |
566 | | - | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
567 | 571 | | |
568 | 572 | | |
569 | 573 | | |
| |||
582 | 586 | | |
583 | 587 | | |
584 | 588 | | |
585 | | - | |
| 589 | + | |
| 590 | + | |
586 | 591 | | |
587 | 592 | | |
588 | 593 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
137 | 137 | | |
138 | 138 | | |
139 | 139 | | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
140 | 166 | | |
141 | 167 | | |
142 | 168 | | |
| |||
0 commit comments