Skip to content

feat: GPU HNSW - #51786

Open
premal wants to merge 14 commits into
milvus-io:masterfrom
6si:gpu-hnsw
Open

feat: GPU HNSW#51786
premal wants to merge 14 commits into
milvus-io:masterfrom
6si:gpu-hnsw

Conversation

@premal

@premal premal commented Jul 23, 2026

Copy link
Copy Markdown

Summary

Adds the GPU_HNSW / GPU_HNSW_SQ index types to Milvus, backed by the faiss-native faiss::gpu::GpuIndexHNSW (through knowhere). A persisted HNSW segment is uploaded to the GPU at load time and searched on device — no GPU build path, the CPU-built graph is reused as-is. Supports fp32 / fp16 / bf16 / int8 storage across L2 / IP / COSINE.

Pairs with knowhere (GPU HNSW + accurate int8 cosine) and faiss (GpuIndexHNSW, facebookresearch/faiss#5459). Motivation: CPU HNSW plateaus at ~1,600 vec/s per node (DRAM-bandwidth bound) on our INT8_VECTOR dim-384 COSINE workload at ~500M+ rows; this gives the existing HNSW graph a GPU search path with recall parity to CPU.

What's in this PR

Index registration & param validation

  • internal/util/indexparamcheck/gpu_hnsw_checker.go (+ conf_adapter_mgr.go, index_type.go): GPU_HNSW / GPU_HNSW_SQ index-param validation.
  • client/index/common.go, pkg/util/indexparams/index_params.go: index-type plumbing.

Load-time GPU override

  • internal/querynodev2/segments/segment.go, services.go, internal/querycoordv2/task/executor.go: persisted type stays HNSW; a per-segment override_index_type swaps to the GPU index at load. The collection-level IndexInfoList is unchanged by design (a cluster can serve the same collection on CPU or GPU nodes).

GPU VRAM admission (the key resource change)

GPU segments must be admitted against device memory, not host RAM. This threads a GPU-memory estimate from knowhere through the C ABI into the Go loader:

  • internal/core/src/common/resource_c.h: add uint64_t gpu_memory_cost;
  • internal/core/src/index/IndexFactory.cpp: request.gpu_memory_cost = res.gpuMemoryCost;
  • internal/querynodev2/segments/segment_loader.go: GPU admission uses the GPU estimate with a host-RAM fallback, and reserves host memory separately:
gpuMemoryCost := estimateResult.GpuMemoryCost
if gpuMemoryCost == 0 {
    gpuMemoryCost = estimateResult.MaxMemoryCost   // fallback: conservative
}
fieldGpuMemorySize = append(fieldGpuMemorySize, gpuMemoryCost)

The reservation is conservative (≈2× file size) so admission never under-reserves; measured int8-cosine device usage is ≈1.7× file, so a node stops/​rebalances before it can OOM.

knowhere pin

internal/core/thirdparty/knowhere/CMakeLists.txt: pins knowhere to the GPU-HNSW ref (vanilla faiss GpuIndexHNSW + accurate int8 cosine + faiss review fixes). Pinned to an immutable SHA rather than a branch. For an upstream merge this would be repointed to a tagged knowhere release once the knowhere PR lands (faiss → knowhere → milvus order).

Tests

  • internal/util/indexparamcheck/gpu_hnsw_checker_test.go, pkg/util/indexparams/index_params_test.go: param validation.
  • internal/querynodev2/segments/segment_loader_test.go: GPU admission sizing.
  • tests/python_client/testcases/indexes/{idx_gpu_hnsw,idx_gpu_hnsw_sq,test_gpu_hnsw,test_gpu_hnsw_sq}.py + test_gpu_hnsw_e2e.py: E2E build/load/search across dtypes and metrics.

Design notes / docs

  • Design doc: docs/design-docs/design_docs/20260619-gpu-hnsw.md (single consolidated MEP — architecture, native fp32/fp16/bf16/int8 storage, GPU VRAM admission, CPU-parity filtered search).
  • Load-time override keeps persisted segments portable between CPU and GPU query nodes.
  • GPU resource accounting is generic; only GPU_HNSW populates gpuMemoryCost under the current knowhere pin.

Hardware / build

  • NVIDIA GPU compute capability 7.0+ (Volta+), CUDA 12.x+; GPU Docker image (build/docker/milvus/gpu/ubuntu22.04/Dockerfile).

Related Issue

premal and others added 8 commits July 22, 2026 05:43
Wire up GPU_HNSW / GPU_HNSW_SQ index types backed by the faiss-native
faiss::gpu::GpuIndexHNSW (via knowhere), with load-time override of persisted
HNSW segments to the GPU index, VRAM admission via Resource.maxMemoryCost, and
a gpu_hnsw index-param checker. Pins knowhere to gpu-hnsw@97f933f3 (vanilla
faiss GpuIndexHNSW + accurate int8 cosine + faiss review fixes).

- IndexFactory: maxMemoryCost>0 drives GPU segment VRAM admission (only
  GPU_HNSW populates it under the current pin).
- segment_loader: GPU segment admission sizing.
- indexparamcheck: gpu_hnsw_checker (GPU_HNSW / GPU_HNSW_SQ).
- Design docs: gpu-hnsw, ocq-removal (FP16/BF16 GPU-validated), filtered-search
  (refreshed source anchors). Python E2E + index test cases.

Clean single-commit re-creation off current master (replaces the stacked
gpu-hnsw-faiss-native branch).

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Repin knowhere gpu-hnsw 97f933f3 -> 9c0af31f, which re-vendors the faiss #11
review fix: the native int8 DP4A layer-0 path is now selected on a per-search
staged flag rather than a leftover pooled d_queries_i8 buffer, so a fp32-query
search reusing a scratch slot can't score against stale int8 query data.

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…-only today

Comment-only. Addresses the Devin/Magnitude review note on IndexFactory's
VectorMemIndexLoadResource: document that GPU_HNSW is currently the sole index
type populating res.maxMemoryCost (> 0), so the peak-based admission branch is
effectively GPU_HNSW-scoped today while remaining generic for any future index
that sets maxMemoryCost. No behavior change; does not affect the v4 image
(gpu-hnsw-faiss-native-v4, built from b9d094e).

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…9d92eb1)

Repoints KNOWHERE_VERSION to 99d92eb1, which re-vendors the faiss GpuIndexHNSW
on-device label-conversion kernel (search() path). No behavior change for
milvus: the querynode uses the searchHost/searchHostInt8 path, which was
already free of the D2H->H2D label round-trip. Keeps the pin in sync with
faiss gpu-hnsw dce385d9.

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sync gpu-hnsw with upstream milvus master (78 commits). Re-pin knowhere to
6si/knowhere 2959fe3d (gpu-hnsw merged up to knowhere main), which now carries
the expected<T> iterator error-code interface (milvus-io#1699) that master's segcore
adaptation depends on. Resolved segment_loader.go by adopting master's
prepareIndexLoadParams() helper (our side only added a doc comment; the helper
preserves DiskANN/bitmap + AppendPrepareLoadParams logic).

Co-Authored-By: Devin <devin@cognition.ai>
Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…here 0d01cac0

Knowhere now reports a device (VRAM) footprint distinct from the host
transient peak. Plumb it through: add gpu_memory_cost to the C
LoadResourceRequest, set it from Resource::gpuMemoryCost in
VectorIndexLoadResource, expose it as ResourceEstimate.GpuMemoryCost, and use
it for the querynode GPU admission reservation (fieldGpuMemorySize), falling
back to MaxMemoryCost when a GPU index does not report a distinct device cost.

Without this, GPU admission reused the host transient MaxMemoryCost (raised to
4x file for int8-cosine to fix the host OOM), which would over-reserve VRAM
(~2.3x the real ~1.7x/file device growth) and falsely reject GPU loads.

Re-pin knowhere to 0d01cac0 (chunked cosine reconstruct + metric-aware host
estimate + separate gpuMemoryCost + faiss_gpu_hnsw PIC build fix).

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Fold the filtered-search MEP and the OCQ-removal/native-fp16-bf16 decision
record into 20260619-gpu-hnsw.md and drop the two separate docs. Update the
two python test docstrings that referenced the removed filtered-search doc.

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@sre-ci-robot
sre-ci-robot requested review from aoiasd and czs007 July 23, 2026 23:59
@sre-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: premal
To complete the pull request process, please assign xiaofan-luan after the PR has been reviewed.
You can assign the PR to them by writing /assign @xiaofan-luan in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot added area/compilation size/XXL Denotes a PR that changes 1000+ lines. area/test sig/testing labels Jul 23, 2026
@mergify mergify Bot added the dco-passed DCO check passed. label Jul 23, 2026
@mergify

mergify Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@premal

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

  1. Title Format: The PR title must begin with one of these prefixes:
  • feat: for introducing a new feature.
  • fix: for bug fixes.
  • enhance: for improvements to existing functionality.
  • test: for add tests to existing functionality.
  • doc: for modifying documentation.
  • auto: for the pull request from bot.
  • build(deps): for dependency updates from Dependabot.
  1. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly 

Please review and update your PR to comply with these guidelines.

@sre-ci-robot

Copy link
Copy Markdown
Contributor

[ci-v2-notice]
Notice: New ci-v2 system is enabled for this PR.

To rerun ci-v2 checks, comment with:

  • /ci-rerun-code-check // for ci-v2/code-check
  • /ci-rerun-code-check-macos // for Code Checker MacOS (GitHub Actions)
  • /ci-rerun-build // for ci-v2/build
  • /ci-rerun-build-all // for ci-v2/build-all (multi-arch builds)
  • /ci-rerun-buildenv // for ci-v2/build-env (build milvus-env builder images; update .env after the new tag is ready)
  • /ci-rerun-ut-integration // for ci-v2/ut-integration, will rerun ci-v2/build
  • /ci-rerun-ut-go // for ci-v2/ut-go, will rerun ci-v2/build
  • /ci-rerun-ut-cpp // for ci-v2/ut-cpp
  • /ci-rerun-ut // for all ci-v2/ut-integration, ci-v2/ut-go, ci-v2/ut-cpp, will rerun ci-v2/build
  • /ci-rerun-e2e-default // for ci-v2/e2e-default
  • /ci-rerun-e2e-amd // for ci-v2/e2e-amd (e2e pool dispatcher)
  • /ci-rerun-e2e-dist-wp // for ci-v2/e2e-dist-wp (Tencent distributed woodpecker-service boundary)
  • /ci-rerun-build-ut-cov // for ci-v2/build-ut-cov (build + unit tests in one pipeline)
  • /ci-rerun-gosdk // for ci-v2/go-sdk (Go SDK E2E tests, ARM)
  • /ci-rerun-gosdk-std-wp // for ci-v2/go-sdk-std-wp (Go SDK E2E, standalone + embedded Woodpecker)
  • /ci-rerun-gosdk-dist-wp // for ci-v2/go-sdk-dist-wp (Go SDK E2E, distributed + Woodpecker service)

If you have any questions or requests, please contact @zhikunyao.

@sre-ci-robot

Copy link
Copy Markdown
Contributor

❌ CI Loop Results cfb2fe7

Stage Result Duration Tests
✅ Build SUCCESS 16.5min -
❌ Code-Check FAILURE 3.9min -
❌ UT-Integration SKIPPED - -
❌ UT-GO SKIPPED - -
✅ UT-CPP-Cov SUCCESS 66.7min 8243 total, 8243 passed, 0 failed

Total: 89min | Pipeline | Artifacts
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured)
Total Patch Coverage: 100.0% (2/2 measurable lines, 18 unmeasured)

Failed Test Logs:

premal and others added 2 commits July 24, 2026 07:02
Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@premal premal changed the title GPU HNSW feat: GPU HNSW Jul 24, 2026
@mergify mergify Bot added kind/feature Issues related to feature request from users and removed do-not-merge/invalid-pr-format labels Jul 24, 2026
@mergify

mergify Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@premal Please associate the related issue to the body of your Pull Request. (eg. "issue: #")

…rmat); revert unrelated common_func import tweak

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@sre-ci-robot

Copy link
Copy Markdown
Contributor

✅ CI Loop Results 4493ff9

Stage Result Duration Tests
✅ Build SUCCESS 11.2min -
✅ Code-Check SUCCESS 8.2min -
✅ UT-Integration SUCCESS 24.3min -
✅ UT-GO SUCCESS 22.6min -
✅ UT-CPP-Cov SUCCESS 46.1min 8243 total, 8243 passed, 0 failed

Total: 96min | Pipeline | Artifacts

Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 45.5% (46 hit, 55 miss, 101 measurable lines, 115 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 115 changed lines were unmeasured.
Total Patch Coverage: 46.6% (48/103 measurable lines, 133 unmeasured)

@sre-ci-robot

Copy link
Copy Markdown
Contributor

❌ CI Loop Results 6fd26d9

Stage Result Duration Tests
✅ Build SUCCESS 16.1min -
❌ Code-Check FAILURE 4.5min -
❌ UT-Integration SKIPPED - -
❌ UT-GO SKIPPED - -
✅ UT-CPP-Cov SUCCESS 65.8min 8243 total, 8243 passed, 0 failed

Total: 104min | Pipeline | Artifacts
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured)
Total Patch Coverage: 100.0% (2/2 measurable lines, 18 unmeasured)

Failed Test Logs:

…fp16 re-encode)

Adds the reconciliation Magnitude flagged as an open item: the int8 device
layout (native ~1x vs fp16 re-encode ~1.7x) is decided at load by the
deserialized storage class (HasInverseL2Norms), not the config metric. Explains
why a querynode log showing upload_int8_dataset (1 B/dim, native) and an
isolated cosine-subclass collection re-encoding to fp16 are both correct and
consistent, and why the re-encode is not dead code. No code change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@sre-ci-robot

Copy link
Copy Markdown
Contributor

✅ CI Loop Results 2324c32

Stage Result Duration Tests
✅ Build SUCCESS 10.7min -
✅ Code-Check SUCCESS 7.4min -
✅ UT-Integration SUCCESS 24.1min -
✅ UT-GO SUCCESS 22.6min -
✅ UT-CPP-Cov SUCCESS 45.7min 8243 total, 8243 passed, 0 failed

Total: 73min | Pipeline | Artifacts

Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 45.5% (46 hit, 55 miss, 101 measurable lines, 115 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 115 changed lines were unmeasured.
Total Patch Coverage: 46.6% (48/103 measurable lines, 133 unmeasured)

…ive check)

Live-cluster probe (v7) confirms mpd_v2 is IndexHNSWSQCosine (is_cosine=1 on all
sampled segments, upload_int8_dataset absent) and takes the fp16 re-encode branch
(~1.7x resident / 2.0x estimator peak), matching vram_probe_int8_cos. The native
upload_int8_dataset (1 B/dim) logs come from non-cosine int8 collections, not
mpd_v2. No code change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@sre-ci-robot

Copy link
Copy Markdown
Contributor

✅ CI Loop Results 459c812

Stage Result Duration Tests
✅ Build SUCCESS 10.5min -
✅ Code-Check SUCCESS 6.9min -
✅ UT-Integration SUCCESS 23.9min -
✅ UT-GO SUCCESS 22.6min -
✅ UT-CPP-Cov SUCCESS 45.6min 8243 total, 8243 passed, 0 failed

Total: 73min | Pipeline | Artifacts

Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 45.5% (46 hit, 55 miss, 101 measurable lines, 115 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 115 changed lines were unmeasured.
Total Patch Coverage: 46.6% (48/103 measurable lines, 133 unmeasured)

Consolidate the three duplicated load-stage override loops (QueryNode
LoadSegments handler, segmentLoader.Load, segmentLoader.ReopenSegments)
into one exported, unit-tested helper ApplyLoadStageOverrides, and extract
the numRows fallback (resolveIndexNumRows) and GPU VRAM admission fallback
(gpuAdmissionCost) into small pure helpers so the previously-uncovered
GPU_HNSW glue is directly testable.

Tests added:
- querycoordv2/task: getLoadInfo applies (and, without config, skips) the
  HNSW -> GPU_HNSW load-stage override.
- querynodev2/segments: ApplyLoadStageOverrides override/no-override/nil;
  resolveIndexNumRows fallback (NumRows==0/negative -> segment rows, the
  OOM-admission regression); gpuAdmissionCost device-vs-host fallback.

No behavior change: helpers preserve the exact prior semantics.

Signed-off-by: premal <premal@6sense.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@sre-ci-robot

Copy link
Copy Markdown
Contributor

✅ CI Loop Results 2d7e280

Stage Result Duration Tests
✅ Build SUCCESS 10.2min -
✅ Code-Check SUCCESS 6.0min -
✅ UT-Integration SUCCESS 23.5min -
✅ UT-GO SUCCESS 17.1min -
✅ UT-CPP-Cov SUCCESS 45.1min 8243 total, 8243 passed, 0 failed

Total: 65min | Pipeline | Artifacts

Overall Coverage: 73.1%
Diff Coverage: CPP 100.0% (2 hit, 0 miss, 2 measurable lines, 18 unmeasured) | Go 55.1% (54 hit, 44 miss, 98 measurable lines, 136 unmeasured)
Diff Coverage HTML: view changed lines
Go Patch Warning: WARNING: Go patch coverage is partial; 136 changed lines were unmeasured.
Total Patch Coverage: 56.0% (56/100 measurable lines, 154 unmeasured)

@sre-ci-robot sre-ci-robot added the do-not-merge/need-milestone generate by v2-label-manager label Aug 13, 2026
@sre-ci-robot

Copy link
Copy Markdown
Contributor

[INFO] PR Label Summary by Default
[INFO] Dependent PR check skipped - not required for branch 'master'

[WARNING] Milestone not set

You can set milestone by commenting:
/set-milestone
Example:
/set-milestone 2.5.0

Use /refresh-label to update related check and label manually

@sre-ci-robot sre-ci-robot added the do-not-merge/doc-need-two-approve Gray observation: docs PR needs two distinct approve commenters label Aug 14, 2026
@zhikunyao zhikunyao removed the do-not-merge/need-milestone generate by v2-label-manager label Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compilation area/test dco-passed DCO check passed. do-not-merge/doc-need-two-approve Gray observation: docs PR needs two distinct approve commenters kind/feature Issues related to feature request from users sig/testing size/XXL Denotes a PR that changes 1000+ lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants