feat: GPU HNSW - #51786
Conversation
Wire up GPU_HNSW / GPU_HNSW_SQ index types backed by the faiss-native faiss::gpu::GpuIndexHNSW (via knowhere), with load-time override of persisted HNSW segments to the GPU index, VRAM admission via Resource.maxMemoryCost, and a gpu_hnsw index-param checker. Pins knowhere to gpu-hnsw@97f933f3 (vanilla faiss GpuIndexHNSW + accurate int8 cosine + faiss review fixes). - IndexFactory: maxMemoryCost>0 drives GPU segment VRAM admission (only GPU_HNSW populates it under the current pin). - segment_loader: GPU segment admission sizing. - indexparamcheck: gpu_hnsw_checker (GPU_HNSW / GPU_HNSW_SQ). - Design docs: gpu-hnsw, ocq-removal (FP16/BF16 GPU-validated), filtered-search (refreshed source anchors). Python E2E + index test cases. Clean single-commit re-creation off current master (replaces the stacked gpu-hnsw-faiss-native branch). Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Repin knowhere gpu-hnsw 97f933f3 -> 9c0af31f, which re-vendors the faiss #11 review fix: the native int8 DP4A layer-0 path is now selected on a per-search staged flag rather than a leftover pooled d_queries_i8 buffer, so a fp32-query search reusing a scratch slot can't score against stale int8 query data. Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…-only today Comment-only. Addresses the Devin/Magnitude review note on IndexFactory's VectorMemIndexLoadResource: document that GPU_HNSW is currently the sole index type populating res.maxMemoryCost (> 0), so the peak-based admission branch is effectively GPU_HNSW-scoped today while remaining generic for any future index that sets maxMemoryCost. No behavior change; does not affect the v4 image (gpu-hnsw-faiss-native-v4, built from b9d094e). Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…9d92eb1) Repoints KNOWHERE_VERSION to 99d92eb1, which re-vendors the faiss GpuIndexHNSW on-device label-conversion kernel (search() path). No behavior change for milvus: the querynode uses the searchHost/searchHostInt8 path, which was already free of the D2H->H2D label round-trip. Keeps the pin in sync with faiss gpu-hnsw dce385d9. Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sync gpu-hnsw with upstream milvus master (78 commits). Re-pin knowhere to 6si/knowhere 2959fe3d (gpu-hnsw merged up to knowhere main), which now carries the expected<T> iterator error-code interface (milvus-io#1699) that master's segcore adaptation depends on. Resolved segment_loader.go by adopting master's prepareIndexLoadParams() helper (our side only added a doc comment; the helper preserves DiskANN/bitmap + AppendPrepareLoadParams logic). Co-Authored-By: Devin <devin@cognition.ai> Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…here 0d01cac0 Knowhere now reports a device (VRAM) footprint distinct from the host transient peak. Plumb it through: add gpu_memory_cost to the C LoadResourceRequest, set it from Resource::gpuMemoryCost in VectorIndexLoadResource, expose it as ResourceEstimate.GpuMemoryCost, and use it for the querynode GPU admission reservation (fieldGpuMemorySize), falling back to MaxMemoryCost when a GPU index does not report a distinct device cost. Without this, GPU admission reused the host transient MaxMemoryCost (raised to 4x file for int8-cosine to fix the host OOM), which would over-reserve VRAM (~2.3x the real ~1.7x/file device growth) and falsely reject GPU loads. Re-pin knowhere to 0d01cac0 (chunked cosine reconstruct + metric-aware host estimate + separate gpuMemoryCost + faiss_gpu_hnsw PIC build fix). Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Fold the filtered-search MEP and the OCQ-removal/native-fp16-bf16 decision record into 20260619-gpu-hnsw.md and drop the two separate docs. Update the two python test docstrings that referenced the removed filtered-search doc. Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: premal The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Invalid PR Title Format Detected Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:
Required Title Structure: Where Example: Please review and update your PR to comply with these guidelines. |
|
[ci-v2-notice] To rerun ci-v2 checks, comment with:
If you have any questions or requests, please contact @zhikunyao. |
❌ CI Loop Results
|
| Stage | Result | Duration | Tests |
|---|---|---|---|
| ✅ Build | SUCCESS | 16.5min | - |
| ❌ Code-Check | FAILURE | 3.9min | - |
| ❌ UT-Integration | SKIPPED | - | - |
| ❌ UT-GO | SKIPPED | - | - |
| ✅ UT-CPP-Cov | SUCCESS | 66.7min | 8243 total, 8243 passed, 0 failed |
Total: 89min | Pipeline | Artifacts
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured)
Total Patch Coverage: 100.0% (2/2 measurable lines, 18 unmeasured)
Failed Test Logs:
- Code-Check: view log
Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
@premal Please associate the related issue to the body of your Pull Request. (eg. "issue: #") |
…rmat); revert unrelated common_func import tweak Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
✅ CI Loop Results
|
| Stage | Result | Duration | Tests |
|---|---|---|---|
| ✅ Build | SUCCESS | 11.2min | - |
| ✅ Code-Check | SUCCESS | 8.2min | - |
| ✅ UT-Integration | SUCCESS | 24.3min | - |
| ✅ UT-GO | SUCCESS | 22.6min | - |
| ✅ UT-CPP-Cov | SUCCESS | 46.1min | 8243 total, 8243 passed, 0 failed |
Total: 96min | Pipeline | Artifacts
Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 45.5% (46 hit, 55 miss, 101 measurable lines, 115 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 115 changed lines were unmeasured.
Total Patch Coverage: 46.6% (48/103 measurable lines, 133 unmeasured)
❌ CI Loop Results
|
| Stage | Result | Duration | Tests |
|---|---|---|---|
| ✅ Build | SUCCESS | 16.1min | - |
| ❌ Code-Check | FAILURE | 4.5min | - |
| ❌ UT-Integration | SKIPPED | - | - |
| ❌ UT-GO | SKIPPED | - | - |
| ✅ UT-CPP-Cov | SUCCESS | 65.8min | 8243 total, 8243 passed, 0 failed |
Total: 104min | Pipeline | Artifacts
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured)
Total Patch Coverage: 100.0% (2/2 measurable lines, 18 unmeasured)
Failed Test Logs:
- Code-Check: view log
…fp16 re-encode) Adds the reconciliation Magnitude flagged as an open item: the int8 device layout (native ~1x vs fp16 re-encode ~1.7x) is decided at load by the deserialized storage class (HasInverseL2Norms), not the config metric. Explains why a querynode log showing upload_int8_dataset (1 B/dim, native) and an isolated cosine-subclass collection re-encoding to fp16 are both correct and consistent, and why the re-encode is not dead code. No code change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
✅ CI Loop Results
|
| Stage | Result | Duration | Tests |
|---|---|---|---|
| ✅ Build | SUCCESS | 10.7min | - |
| ✅ Code-Check | SUCCESS | 7.4min | - |
| ✅ UT-Integration | SUCCESS | 24.1min | - |
| ✅ UT-GO | SUCCESS | 22.6min | - |
| ✅ UT-CPP-Cov | SUCCESS | 45.7min | 8243 total, 8243 passed, 0 failed |
Total: 73min | Pipeline | Artifacts
Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 45.5% (46 hit, 55 miss, 101 measurable lines, 115 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 115 changed lines were unmeasured.
Total Patch Coverage: 46.6% (48/103 measurable lines, 133 unmeasured)
…ive check) Live-cluster probe (v7) confirms mpd_v2 is IndexHNSWSQCosine (is_cosine=1 on all sampled segments, upload_int8_dataset absent) and takes the fp16 re-encode branch (~1.7x resident / 2.0x estimator peak), matching vram_probe_int8_cos. The native upload_int8_dataset (1 B/dim) logs come from non-cosine int8 collections, not mpd_v2. No code change. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
✅ CI Loop Results
|
| Stage | Result | Duration | Tests |
|---|---|---|---|
| ✅ Build | SUCCESS | 10.5min | - |
| ✅ Code-Check | SUCCESS | 6.9min | - |
| ✅ UT-Integration | SUCCESS | 23.9min | - |
| ✅ UT-GO | SUCCESS | 22.6min | - |
| ✅ UT-CPP-Cov | SUCCESS | 45.6min | 8243 total, 8243 passed, 0 failed |
Total: 73min | Pipeline | Artifacts
Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 45.5% (46 hit, 55 miss, 101 measurable lines, 115 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 115 changed lines were unmeasured.
Total Patch Coverage: 46.6% (48/103 measurable lines, 133 unmeasured)
Consolidate the three duplicated load-stage override loops (QueryNode LoadSegments handler, segmentLoader.Load, segmentLoader.ReopenSegments) into one exported, unit-tested helper ApplyLoadStageOverrides, and extract the numRows fallback (resolveIndexNumRows) and GPU VRAM admission fallback (gpuAdmissionCost) into small pure helpers so the previously-uncovered GPU_HNSW glue is directly testable. Tests added: - querycoordv2/task: getLoadInfo applies (and, without config, skips) the HNSW -> GPU_HNSW load-stage override. - querynodev2/segments: ApplyLoadStageOverrides override/no-override/nil; resolveIndexNumRows fallback (NumRows==0/negative -> segment rows, the OOM-admission regression); gpuAdmissionCost device-vs-host fallback. No behavior change: helpers preserve the exact prior semantics. Signed-off-by: premal <premal@6sense.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
✅ CI Loop Results
|
| Stage | Result | Duration | Tests |
|---|---|---|---|
| ✅ Build | SUCCESS | 10.2min | - |
| ✅ Code-Check | SUCCESS | 6.0min | - |
| ✅ UT-Integration | SUCCESS | 23.5min | - |
| ✅ UT-GO | SUCCESS | 17.1min | - |
| ✅ UT-CPP-Cov | SUCCESS | 45.1min | 8243 total, 8243 passed, 0 failed |
Total: 65min | Pipeline | Artifacts
Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 55.1% (54 hit, 44 miss, 98 measurable lines, 136 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 136 changed lines were unmeasured.
Total Patch Coverage: 56.0% (56/100 measurable lines, 154 unmeasured)
|
[INFO] PR Label Summary by Default [WARNING] Milestone not set
You can set milestone by commenting: Use /refresh-label to update related check and label manually |
Summary
Adds the
GPU_HNSW/GPU_HNSW_SQindex types to Milvus, backed by the faiss-nativefaiss::gpu::GpuIndexHNSW(through knowhere). A persisted HNSW segment is uploaded to the GPU at load time and searched on device — no GPU build path, the CPU-built graph is reused as-is. Supports fp32 / fp16 / bf16 / int8 storage across L2 / IP / COSINE.Pairs with knowhere (GPU HNSW + accurate int8 cosine) and faiss (
GpuIndexHNSW, facebookresearch/faiss#5459). Motivation: CPU HNSW plateaus at ~1,600 vec/s per node (DRAM-bandwidth bound) on ourINT8_VECTORdim-384 COSINE workload at ~500M+ rows; this gives the existing HNSW graph a GPU search path with recall parity to CPU.What's in this PR
Index registration & param validation
internal/util/indexparamcheck/gpu_hnsw_checker.go(+conf_adapter_mgr.go,index_type.go):GPU_HNSW/GPU_HNSW_SQindex-param validation.client/index/common.go,pkg/util/indexparams/index_params.go: index-type plumbing.Load-time GPU override
internal/querynodev2/segments/segment.go,services.go,internal/querycoordv2/task/executor.go: persisted type staysHNSW; a per-segmentoverride_index_typeswaps to the GPU index at load. The collection-levelIndexInfoListis unchanged by design (a cluster can serve the same collection on CPU or GPU nodes).GPU VRAM admission (the key resource change)
GPU segments must be admitted against device memory, not host RAM. This threads a GPU-memory estimate from knowhere through the C ABI into the Go loader:
internal/core/src/common/resource_c.h: adduint64_t gpu_memory_cost;internal/core/src/index/IndexFactory.cpp:request.gpu_memory_cost = res.gpuMemoryCost;internal/querynodev2/segments/segment_loader.go: GPU admission uses the GPU estimate with a host-RAM fallback, and reserves host memory separately:The reservation is conservative (≈2× file size) so admission never under-reserves; measured int8-cosine device usage is ≈1.7× file, so a node stops/rebalances before it can OOM.
knowhere pin
internal/core/thirdparty/knowhere/CMakeLists.txt: pins knowhere to the GPU-HNSW ref (vanilla faissGpuIndexHNSW+ accurate int8 cosine + faiss review fixes). Pinned to an immutable SHA rather than a branch. For an upstream merge this would be repointed to a tagged knowhere release once the knowhere PR lands (faiss → knowhere → milvus order).Tests
internal/util/indexparamcheck/gpu_hnsw_checker_test.go,pkg/util/indexparams/index_params_test.go: param validation.internal/querynodev2/segments/segment_loader_test.go: GPU admission sizing.tests/python_client/testcases/indexes/{idx_gpu_hnsw,idx_gpu_hnsw_sq,test_gpu_hnsw,test_gpu_hnsw_sq}.py+test_gpu_hnsw_e2e.py: E2E build/load/search across dtypes and metrics.Design notes / docs
docs/design-docs/design_docs/20260619-gpu-hnsw.md(single consolidated MEP — architecture, native fp32/fp16/bf16/int8 storage, GPU VRAM admission, CPU-parity filtered search).GPU_HNSWpopulatesgpuMemoryCostunder the current knowhere pin.Hardware / build
build/docker/milvus/gpu/ubuntu22.04/Dockerfile).Related Issue