Skip to content

fix(index): support HNSW segment merge#7178

Merged
Xuanwo merged 2 commits into
mainfrom
xuanwo/hnsw-merge
Jun 9, 2026
Merged

fix(index): support HNSW segment merge#7178
Xuanwo merged 2 commits into
mainfrom
xuanwo/hnsw-merge

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

HNSW distributed index builds can currently create multiple index segments, but the physical merge path did not preserve a usable HNSW graph in the merged root index.

This PR rebuilds the HNSW graph from merged auxiliary storage during segment merge for IVF_HNSW_FLAT, IVF_HNSW_PQ, and IVF_HNSW_SQ. It preserves merged IVF/storage metadata, validates that source HNSW build parameters are consistent before rebuilding, and rejects ambiguous merges instead of producing order-dependent graphs.

Fixes #6309.

@github-actions github-actions Bot added the bug Something isn't working label Jun 9, 2026
@Xuanwo Xuanwo marked this pull request as ready for review June 9, 2026 10:36
@github-actions github-actions Bot added the A-index Vector index, linalg, tokenizer label Jun 9, 2026
@codecov

codecov Bot commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.43353% with 85 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance/src/index/vector/ivf.rs 71.66% 35 Missing and 50 partials ⚠️

📢 Thoughts on this report? Let us know!

@Xuanwo Xuanwo merged commit 07efb90 into main Jun 9, 2026
30 checks passed
@Xuanwo Xuanwo deleted the xuanwo/hnsw-merge branch June 9, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-index Vector index, linalg, tokenizer bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tracking: Distributed Indexes Search

2 participants