4.0.0-rc.8

JVector Release Notes

Version: 4.0.0-rc.7 - 4.0.0-rc.8

---

New Features and Enhancements

Fused Product Quantization (Fused PQ)

Description
Fused Product Quantization (Fused PQ) is a performance optimization that embeds compressed Product Quantization (PQ) codes directly into the graph index structure alongside each node's neighbor lists. This eliminates the need for separate lookups to retrieve compressed vectors during graph traversal, significantly improving query performance by reducing memory access overhead. The feature stores PQ-encoded neighbor vectors inline with the graph edges, enabling faster approximate similarity scoring during search operations. By embedding the compressed neighbor vectors into the graph index itself, Fused PQ eliminates the need to maintain a separate in-memory data structure for PQ-encoded vectors, reducing heap memory usage while maintaining fast approximate similarity search performance.​

Purpose / Impact

• Reduces memory usage for large-scale vector datasets
• Improves cache locality during graph traversal
• Enables higher writer scalability for large / high-dimensional vector workloads

How to Enable
To enable Fused PQ when writing an on-disk graph index:

1. Create the FusedPQ feature by passing your graph's max degree and a ProductQuantization compressor to the constructor:

   var fusedPQFeature = new FusedPQ(graph.maxDegree(), pq);

2. Add it to your OnDiskGraphIndexWriter builder:

   var writer = new OnDiskGraphIndexWriter.Builder(graph, outputPath)
       .with(fusedPQFeature)
       .build();

3. Provide a state supplier during the write phase that includes the graph view and PQ vectors:

   Map<FeatureId, IntFunction<Feature.State>> writeSuppliers = new EnumMap<>(FeatureId.class);
   writeSuppliers.put(FeatureId.FUSED_PQ, ordinal -> new FusedPQ.State(view, pqVectors, ordinal));
   writer.write(writeSuppliers);

Notes

• Fused PQ requires a 256-cluster ProductQuantization compressor. The feature automatically embeds compressed neighbor vectors inline with the graph structure during the write operation.
• To enable the FUSED_PQ feature, we introduced the new version 6 file format for our graph indices.

---

Parallel Graph Index Construction

Description
OnDiskParallelGraphIndexWriter significantly accelerates graph index construction by addressing the disk I/O bottleneck that limits the serial OnDiskGraphIndexWriter. This implementation uses asynchronous file I/O with multiple worker threads to write graph records in parallel, with parallelism automatically determined by available system resources (or configurable via builder options). By parallelizing both record building and disk writes while maintaining correct ordering, this approach dramatically reduces the time required to persist large graph indexes to disk.

Purpose / Impact

• Eliminates i/o bottleneck in on disk graph construction
• Maintains backwards compatibility for existing clients of the JVector library

How to Enable
To enable parallel graph index writes, simply use OnDiskParallelGraphIndexWriter.Builder instead of OnDiskGraphIndexWriter.Builder:

Basic usage (uses default parallelism based on available processors):

try (var writer = new OnDiskParallelGraphIndexWriter.Builder(graph, outputPath)
        .with(features...)
        .build()) {
    writer.write(featureSuppliers);
}

Advanced configuration:

try (var writer = new OnDiskParallelGraphIndexWriter.Builder(graph, outputPath)
        .with(features...)
        .withParallelWorkerThreads(8)           // Optional: specify thread count (0 = auto)
        .withParallelDirectBuffers(true)        // Optional: use direct ByteBuffers for better performance
        .build()) {
    writer.write(featureSuppliers);
}

The parallel writer is a drop-in replacement for the standard writer with the same API, automatically leveraging multiple threads and asynchronous I/O to accelerate the write process.

Notes

• Currently still marked as @experimental
• Includes deprecation of method

public synchronized void writeInline(int ordinal, Map<FeatureId, Feature.State> stateMap)

in favor of the more descriptive method

public synchronized void writeFeaturesInline(int ordinal, Map<FeatureId, Feature.State> stateMap)

Related Issues

• 579

---

Documentation and Tutorials

Description
Detailed javadoc added for all JVector components. Quickstart tutorials added to jvector-examples including documentation and code.

Purpose / Impact

• Provide better documentation for JVector and its components
• Give new users an entry point showing how to use JVector

---

Improved DataSet Handling

Description
Revision of how datasets are handled internally within JVector for acquisition and representation of vector data. Includes overhaul of the loading process, better logging and error handling, and virtualization and metadata handling.

Purpose / Impact

• Make datasets easier to find, work with, and define through metadata for internal testing
• Resolves several issues with regression and inter-release comparisons

Notes

• Used internally by JVector Bench classes, no client impact

---

Testing Enhancements

Description
Enhancements to the JVector testing infrastructure:

• On disk index cache added for Grid benchmark harness
• Logging subsystem overhaul
• New JMH tests
• Test results now include metrics for nodes visited, heap usage, disk usage, PQ Distance

Purpose / Impact

• Faster testing cycle
• Better comprehension of test results
• new metrics to compare inter-release

Notes

• Used internally by JVector, no client impact

Related Issues

• 615
• 616

---

Bug Fixes and Issue Resolutions

Fix: NullPointerException in OnDiskGraphIndex#ramBytesUsed

Problem
Now that we lazily load the inMemoryNeighbors and the inMemoryFeatures, we need to handle the case in OnDiskGraphIndex where they are null or have values that are null when the ramBytesUsed() method is called.

Resolution
Added appropriate null checks and safeguards to ensure ramBytesUsed() can be safely invoked in all valid states.

Related Issues

• #586

---

Fix: Protection Against Invalid Ordinal Mappings

Problem
JVector relies on the calling source code to pass in ordinal maps constructed outside of the JVector library. Improper or inconsistent ordinal mappings can lead to failures when the Graph is built or incorrect indexing or search results.

Resolution
Added safeguards to detect invalid ordinal mappings.

Notes
Full validation of ordinal mapping requires iterating over the entire set of ordinals and can be a costly operation. This safeguard will only be activated if debug logging is enabled or if System.getProperties().containsKey("VECTOR_DEBUG")

Related Issues

• 568

---

Fix: extractTrainingVectors may produce more than MAX_PQ_TRAINING_SET_SIZE vectors

Problem
extractTrainingVectors could return more vectors than the intended maximum (MAX_PQ_TRAINING_SET_SIZE), leading to excessive memory usage during PQ training.

Resolution
Uses floyd's random sampling algorithm to select random training vectors from the RandomAccessVectorValues. The solution has two phases. The first is to select MAX_PQ_TRAINING_SET_SIZE random ordinals. Then, it maps those ordinals to vectors.

Related Issues

• 590

---

4.0.0-rc.6

JVector 4.0.0 Release Candidates 1-6: Combined Release Notes

Overview

This document covers the major features, improvements, and changes introduced across JVector versions 4.0.0-rc.1 through 4.0.0-rc.6, released between July 2, 2025 and November 18, 2025.

Major Features

Graph Index Architecture Improvements

New Graph Index Interfaces (rc.4)

• MutableGraphIndex and ImmutableGraphIndex interfaces - Introduced new interface hierarchy to better separate mutable and immutable graph operations, improving API clarity and type safety
• Heap Graph Reconstruction - Added ability to reconstruct in-memory heap graphs from on-disk graph indexes, enabling flexible graph manipulation and analysis workflows

Building Graphs with New Interfaces (rc.4)

Using MutableGraphIndex and ImmutableGraphIndex

// Build a mutable graph
GraphIndexBuilder builder = new GraphIndexBuilder(
    vectorValues,
    similarityFunction,
    M,
    beamWidth,
    neighborOverflow,
    alpha,
    addHierarchy
);

// Get the mutable interface
MutableGraphIndex mutableGraph = builder.getGraph();

// Complete and get immutable interface
ImmutableGraphIndex immutableGraph = builder.complete();

// Check if graph is hierarchical (rc.6)
if (immutableGraph.isHierarchical()) {
    System.out.println("Graph has HNSW-style hierarchy");
}

// Get dimension (rc.5)
int dimension = immutableGraph.getDimension();

Reconstructing Heap Graph from Disk (rc.4)

// Load an on-disk graph
OnDiskGraphIndex diskGraph = OnDiskGraphIndex.load(
    readerSupplier,
    0  // start offset
);

// Reconstruct as heap graph for manipulation
OnHeapGraphIndex heapGraph = OnHeapGraphIndex.fromDiskGraph(
    diskGraph,
    vectorValues
);

Hierarchical Graph Support (rc.6)

• ImmutableGraphIndex.isHierarchical() - Added method to identify hierarchical graph structures, supporting advanced graph topologies

Vector Quantization Enhancements

NVQ (Non-uniform Vector Quantization) Support (rc.5)

• Global Mean Support - Added NVQuantization::create method that accepts a global mean parameter, enabling more flexible quantization strategies
• Reconstruction Error Computation (rc.6) - Implemented computation of reconstruction errors for vector compressors, allowing quality assessment of quantization methods

Creating NVQ with Global Mean (rc.5)

// Create NVQ with a pre-computed global mean
VectorFloat<?> globalMean = computeGlobalMean(vectors);
NVQuantization nvq = NVQuantization.create(
    vectors,
    subspaceCount,
    NVQuantization.BitsPerDimension.EIGHT,
    globalMean  // New parameter in rc.5
);

Computing Reconstruction Errors (rc.6)

// Assess quantization quality by computing reconstruction errors
VectorCompressor<?> compressor = ProductQuantization.compute(vectors, M, clusterCount, centerData);
double[] errors = compressor.computeReconstructionErrors(vectors);
double avgError = Arrays.stream(errors).average().orElse(0.0);
System.out.println("Average reconstruction error: " + avgError);

Performance Optimizations

Product Quantization (PQ) Improvements (rc.3)

• Concurrent Codebook Access - Switched from synchronized to concurrent map for PQ codebook, reducing contention in multi-threaded scenarios
• Partial Sums for Diversity Checks - Created partial sums for PQ codebook to optimize diversity calculations during graph construction
• Specific BuildScoreProvider for Diversity - Added specialized score provider to avoid extra encoding operations during diversity checks
• PQ Ranging Bugfix and Refactoring - Fixed issues and improved code organization in PQ ranging logic

Configuring Product Quantization (PQ)

Using Concurrent Codebook (rc.3)

The concurrent codebook is now used automatically for thread-safe access. No configuration changes needed.

Enabling Anisotropic PQ (rc.3)

// Enable anisotropic quantization with threshold
float anisotropicThreshold = 2.0f; // Parallel cost multiplier
ProductQuantization pq = ProductQuantization.compute(
    vectors,
    M,
    clusterCount,
    centerData,
    anisotropicThreshold  // Set > 0 to enable anisotropic mode
);

In YAML configuration:

compression:
  - type: PQ
    parameters:
      m: 192
      centerData: No
      anisotropicThreshold: 2.0  # Enable anisotropic quantization

Memory and Allocation Optimizations (rc.3)

• Reduced GraphSearcher Allocations - Minimized object allocations in GraphSearcher for better performance
• SimdOps Refactoring - Refactored SimdOps and NativeSimd operations, simplified VectorUtilSupport for cleaner SIMD code

File I/O and Storage

Large File Support (rc.1)

• Chunked Memory-Mapped Reader - Implemented new chunked memory-mapped reader supporting files larger than 2GB, removing previous file size limitations

Using the Chunked Memory-Mapped Reader (rc.1)

The MappedChunkReader automatically handles files larger than 2GB. It's used automatically by the ReaderSupplierFactory:

// The factory automatically selects the appropriate reader
ReaderSupplier supplier = ReaderSupplierFactory.open(indexPath);

// MappedChunkReader is used as fallback for large files
// No special configuration needed - it's transparent to users

Sequential Disk Writer (rc.1)

• File Format Version 5 - Upgraded on-disk format to version 5 with sequential disk writer, improving write performance and reliability

Build and Construction Improvements

Score Provider Enhancements (rc.5)

• RemappedRandomAccessVectorValues - Added to fix BuildScoreProvider::randomAccessScoreProvider, improving score calculation accuracy
• ScoreTracker Fixes - Fixed initialization and reset methods in ScoreTracker for more reliable tracking

Graph Construction (rc.3, rc.4)

• Accurate Memory Estimation - GraphIndexBuilder::addGraphNode now iterates all graph levels to accurately estimate memory usage
• Footer Constants Consolidation - Removed duplicate footer constants from OnDiskSequentialGraphIndexWriter in favor of AbstractGraphIndexWriter

Graph Construction Parameters

Standard Configuration

GraphIndexBuilder builder = new GraphIndexBuilder(
    vectorValues,
    VectorSimilarityFunction.DOT_PRODUCT,
    32,      // M - maximum connections per node
    100,     // beamWidth - search beam size
    1.2f,    // neighborOverflow - temporary overflow ratio
    1.2f,    // alpha - diversity pruning aggressiveness
    true,    // addHierarchy - enable HNSW hierarchy
    true     // refineFinalGraph - second pass refinement (rc.4)
);

Using Compressed Vectors for Building

// Create compression
ProductQuantization pq = ProductQuantization.compute(vectors, M, 256, false);
CompressedVectors compressed = pq.encodeAll(vectors);

// Build with compressed vectors
BuildScoreProvider scoreProvider = BuildScoreProvider.pqBuildScoreProvider(
    similarityFunction,
    pq,
    compressed
);

GraphIndexBuilder builder = new GraphIndexBuilder(
    scoreProvider,
    dimension,
    M,
    beamWidth,
    neighborOverflow,
    alpha,
    addHierarchy,
    refineFinalGraph
);

API Additions

Dimension Access (rc.5)

• getDimension() Method - Added to ImmutableGraphIndex and all implementations, providing easy access to vector dimensionality

Bug Fixes and Stability

Concurrency Fixes (rc.1)

• Cleanup Concurrency Issue - Fixed issue when calling cleanup while concurrently executing searches, preventing race conditions
• Improved Cleanup Efficiency - Enhanced efficiency and memory usage of GraphIndexBuilder.cleanup

Test Improvements (rc.6)

• Heap Graph Reconstruction Tests - Hardened tests for heap graph reconstruction to ensure reliability
• Low Cardinality Filtering - Made thresholds in TestLowCardinalityFiltering tighter for more accurate testing
• Incremental Insertion Test - Temporarily ignored flaky test testIncrementalInsertionFromOnDiskIndex_withNonIdentityOrdinalMapping with TODO for future fix

Testing and Benchmarking

Benchmark Infrastructure (rc.2, rc.3)

• AUX Counters - Added AUX counters and correlated benchmarks for deeper performance analysis
• Throughput Benchmark Improvements - Enhanced throughput benchmark with better metrics and reporting
• PQ Training Benchmark - Added dedicated benchmark for PQ training performance (rc.1)
• YAML Benchmark Configuration - Enabled specifying benchmarks in YAML files for easier configuration (rc.3)
• Regression Testing - Implemented GitHub Actions regression tests with enhancements (rc.3, rc.4)

Dataset Handling (rc.2)

• Dataset Naming Fix - Fixed dataset naming when using default.yml configuration

Documentation and Developer Experience

Contribution Guidelines (rc.3)

• CONTRIBUTIONS.md - Added initial contributions guide
• PR Checklist Template - Created checklist template and streamlined PR process

Changelog Automation (rc.2)

• GitHub Actions Changelog - Implemented new changelog automation via manual GitHub Actions workflow
• Changelog Generation - Enabled changelog generation using labels

CI/CD Improvements (rc.4)

• GitHub Actions Enhancements - Various improvements to CI/CD pipeline
• Changelog Headers - Fixed changelog header formatting

YAML Benchmark Configuration (rc.3)

Specifying Benchmarks in YAML

Create or modify run.yml:

yamlSchemaVersion: 1

# Specify which benchmarks to run
benchmarks:
  throughput: [AVG, MEDIAN, MAX]
  latency: [AVG, STD, P999]
  count: [visited, expanded]
...