Add RVV SIMD specialization for scalar quantizer distance compute - #5539
Open
anthony-zy wants to merge 1 commit into
Open
Add RVV SIMD specialization for scalar quantizer distance compute#5539anthony-zy wants to merge 1 commit into
anthony-zy wants to merge 1 commit into
Conversation
> Cmd: 1M 768 <SQ> 32 100 16 1 1000 1 <metric>
> metric: **Search ms/query** (lower_is_better)
| 序号 | QT_TYPE | Metric | F1(ms) | SQ1(ms) | F2(ms) | SQ2(ms) | Δ_SQ% | Δ_F% |
|------|-------------------------|--------|--------|---------|--------|---------|---------|--------|
| 01 | QT_4bit | L2 | 1.841 | 6.231 | 1.824 | 1.450 | -76.73% | -0.92% |
| 02 | QT_4bit | IP | 1.652 | 6.086 | 1.674 | 1.273 | -79.08% | +1.33% |
| 03 | QT_4bit_uniform | L2 | 1.806 | 2.560 | 1.806 | 1.135 | -55.66% | +0.00% |
| 04 | QT_4bit_uniform | IP | 1.612 | 6.007 | 1.597 | 1.244 | -79.29% | -0.93% |
| 05 | QT_6bit | L2 | 1.801 | 5.701 | 1.805 | 1.500 | -73.69% | +0.22% |
| 06 | QT_6bit | IP | 1.608 | 5.706 | 1.619 | 1.249 | -78.11% | +0.68% |
| 07 | QT_8bit | L2 | 1.821 | 3.736 | 1.907 | 1.394 | -62.69% | +4.72% |
| 08 | QT_8bit | IP | 1.704 | 3.378 | 1.730 | 1.277 | -62.20% | +1.53% |
| 09 | QT_8bit_direct | L2 | 1.977 | 0.280 | 1.947 | 0.054 | -80.71% | -1.52% |
| 10 | QT_8bit_direct | IP | 1.709 | 0.458 | 1.740 | 0.099 | -78.38% | +1.81% |
| 11 | QT_8bit_direct_signed | L2 | 1.581 | 3.234 | 1.572 | 0.761 | -76.47% | -0.57% |
| 12 | QT_8bit_direct_signed | IP | 1.740 | 0.414 | 1.748 | 0.148 | -64.25% | +0.46% |
| 13 | QT_8bit_uniform | L2 | 1.943 | 3.204 | 1.928 | 0.927 | -71.07% | -0.77% |
| 14 | QT_8bit_uniform | IP | 1.727 | 3.121 | 1.744 | 0.947 | -69.66% | +0.98% |
| 15 | QT_bf16 | L2 | 1.862 | 2.889 | 1.894 | 1.466 | -49.26% | +1.72% |
| 16 | QT_bf16 | IP | 1.619 | 2.846 | 1.793 | 1.204 | -57.70% | +10.75%|
| 17 | QT_fp16 | L2 | 2.011 | 5.691 | 1.950 | 1.570 | -72.41% | -3.03% |
| 18 | QT_fp16 | IP | 1.797 | 5.584 | 1.790 | 1.189 | -78.71% | -0.39% |
---
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds RISC-V Vector Extension SIMD specialization for the scalar
quantizer distance computation, following the codec/quantizer/similarity
specialization already present for other SIMD levels. The kernels use the
runtime vector-length aware m8 vector configuration and are shared across
all RVV hardware widths, instead of assuming a fixed vector length.
Changes
Add RVV specialized L2 / inner-product distance kernels for the scalar
quantizer codecs:
Each kernel reuses the reconstructed (quantizer) L2 / IP formulation that
avoids per-dimension reconstruction, keeping the hot loop in the integer
domain and applying the final scale at the tail.
Performance
Cmd: 1M 768 32 100 16 1 1000 1 (search ms/query, lower is better)
Δ_SQ%: SQ scalar baseline -> RVV;
Δ_F%: Faiss flat baseline -> RVV.