Skip to content

CHANGELOG.md

Latest commit

 

History

History
286 lines (223 loc) · 19.6 KB

CHANGELOG.md

File metadata and controls

286 lines (223 loc) · 19.6 KB
Raw

Changelog

All notable changes to the NVIDIA RAG Blueprint will be documented in this file.

To upgrade your version, see Migration Guide.

Version 2.4.0 (2026-06-03)

This release adds KDB-X (bare kdb+) as a second vector database backend with GPU CAGRA support, validates Blackwell g7e EKS self-hosted LLM deployment, adds NeMo Guardrails helm chart support, and renames embed/rerank model references to the new llama-nemotron branding.

Added

  • KDB-X vector database backend: Connect to a customer-managed kdb+ endpoint over IPC as an alternative to KDB.AI. The KDB-X process loads its .rag.* definitions from kdbx-init.q at q startup (the chart's test pod does this automatically; the adapter fails fast with KdbxNotBootstrappedError if they're missing — it does NOT push them on connect). See docs/change-vectordb-kdbx.md.
  • GPU CAGRA index for KDB-X (Phase 2): kdb+ collections can use NVIDIA cuVS CAGRA GPU- accelerated vector search via KDBX_USE_CUVS=1. Requires APP_VECTORSTORE_ENABLEGPUINDEX=True and APP_VECTORSTORE_ENABLEGPUSEARCH=True on both rag-server and ingestor in addition to kdbx.useCuvs=true in Helm — see Known Issues below.
  • KDB-X cuVS safety guards: CAGRA crashloop protection — kill-switch KDBX_CAGRA_SKIP_PERSISTED_READ=1 (or kdbx.cagra.skipPersistedRead=true) to recover from a corrupted .cagra blob by rebuilding from stored vectors. GPU stamp (_cagrastamp) and readiness canary verify cuVS health before the pod accepts traffic.
  • NeMo Guardrails helm chart: Added helm chart support for NeMo Guardrails (upstream had docker-compose only). Working end-to-end (safe → answer, unsafe → blocked).
  • Blackwell g7e EKS self-hosted LLM: Validated llama-3.3-nemotron-super-49b-v1.5 (FP8, TP1) on g7e (RTX PRO 6000 Blackwell, 96 GB). Reference: deploy/EKS/g7e-llm-nodegroup.yaml and deploy/EKS/rag-values-llm-selfhost-g7e.yaml.
  • KDB-X per-collection metric persistence: Collection metric (L2 or CS) is stamped at creation and survives pod restarts. Configurable via KDBX_METRIC env (default: L2).
  • Airgap NIM weight-cache overlay: deploy/EKS/rag-values-nim-cache.yaml provisions PVCs for pre-cached NIM model weights in airgapped EKS deployments.

Changed

  • Breaking (KDB-X): Provisioning model changed from adapter-push (WS-PROVISION) to server startup-load. kdbx-init.q must now be loaded by q at process startup: q /opt/kx/conf/kdbx-init.q -p <port>. The helm chart does this automatically. Customer-managed KDB-X: add this to your q launch command. The adapter no longer bundles or pushes kdbx-init.q; it raises KdbxNotBootstrappedError if the server is not pre-provisioned. (kdbx-init.q ships at deploy/helm/nvidia-blueprint-rag/files/kdbx/kdbx-init.q.)
  • Renamed embed model: nvidia/llama-3.2-nv-embedqa-1b-v2 → nvidia/llama-nemotron-embed-1b-v2
  • Renamed rerank model: nvidia/llama-3.2-nv-rerankqa-1b-v2 → nvidia/llama-nemotron-rerank-1b-v2
  • Renamed VLM embed model: llama-3.2-nemoretriever-1b-vlm-embed-v1 → llama-nemotron-embed-vl-1b-v2
  • Renamed env var KDBX_PORT → KDBX_LISTEN_PORT (breaking change): the Kubernetes Service named kdbx auto-injects KDBX_PORT=tcp://... into pod environments, causing a collision with the listen-port variable. Update any custom configs using KDBX_PORT.

Fixed

  • Fixed non-unique chunk IDs in KDB-X upload_text: IDs were generated per-batch (0..N), causing cross-batch collisions that silently dropped rows in search result joins.

Known Issues {#kdbx-cagra-double-flag}

  • KDB-X CAGRA double-flag requirement: kdbx.useCuvs=true in Helm alone does not activate GPU CAGRA index creation. Both rag-server and ingestor pods must also set APP_VECTORSTORE_ENABLEGPUINDEX=True and APP_VECTORSTORE_ENABLEGPUSEARCH=True via envVars. The deploy/EKS/rag-values-kdbx-cuvs.yaml overlay sets all required values.

Version 2.3.0 (2025-10-14)

This release adds RTX6000 platform support, adds deployment by using NIM operator, improves vector database pluggability with the blueprint, and other changes.

Added

Changed

  • Migrated default LLM model for reflection to llama-3.3-nemotron-super-49b instead of mixtral-8x22b-instruct-v01.
  • Refactored rag-playground code
    • Use React end to end. Next.js dependencies were deprecated.
    • More developer friendly and intuitive look and feel.
    • rag-playground service is renamed to rag-frontend
  • Refactored helm chart support
    • Expanded and reorganized Helm chart configuration, enabling granular control over service components, resource settings, and observability (tracing, metrics).
    • Introduced ConfigMap and service definitions to facilitate improved application deployment flexibility.
    • Implemented refined service account and secret management in Helm templates.
    • Added a new Helm values file for nim-operator to configure LLM model environment and component toggles.

Fixed

  • Fixed support for long audio file ingestion.
  • Fixed support to ingest images without charts/tables.
  • Fixed requirement of rebuilding rag frontend container when LLM model name was changed.

Removed

  • Removed consistency level configuration support for Milvus.
  • Removed EMBEDDING_NIM_ENDPOINT and EMBEDDING_NIM_MODEL_NAME environment variables for nvingest.
  • Removed unused ENABLE_MULTITURN environment variable from rag-server.
  • Removed ENABLE_NEMOTRON_THINKING environment variable from rag-server.

Known Issues

For the full list of known issues, see Known Issues.

Version 2.2.1 (2025-07-22)

This is a minor patch release that updates to the latest nv-ingest-client version 25.6.3 to fix breaking changes introduced by pypdfium. For details, refer to NVIDIA NV Ingest 25.6.3.

Version 2.2.0 (2025-07-08)

This release adds B200 platform support, a native Python API, and major enhancements for multimodal and metadata features. It also improves deployment flexibility and customization across the RAG blueprint.

Added

Changed

  • Changed API schema to support newly added features
    • POST /collections to be deprecated in favour of POST /collection for ingestor-server
    • New endpoint GET /summary added for rag-server
    • Metadata information available as part of GET /collections and GET /documents API
    • Refer to the migration guide for detailed changes at API level
  • Optimized batch mode ingestion support to improve perf for multi user concurrent file upload.

Known Issues

For the full list of known issues, see Known Issues.

Version 2.1.0 (2025-05-13)

This release reduces overall GPU requirement for the deployment of the blueprint. It also improves the performance and stability for both docker and helm based deployments.

Added

  • Added non-blocking async support to upload documents API
    • Added a new field blocking: bool to control this behaviour from client side. Default is set to true
    • Added a new API /status to monitor state or completion status of uploaded docs
  • Helm chart is published on NGC Public registry.
  • Helm chart customization guide is now available for many optional features. For details, refer to NVIDIA RAG Blueprint Documentation.
  • Issues with very large file upload has been fixed.
  • Security enhancements and stability improvements.

Changed

  • Overall GPU requirement reduced to 2xH100/3xA100.
    • Changed default LLM model to llama-3_3-nemotron-super-49b-v1. This reduces overall GPU needed to deploy LLM model to 1xH100/2xA100
    • Changed default GPU needed for all other NIMs (ingestion and reranker NIMs) to 1xH100/1xA100
  • Changed default chunk size to 512 in order to reduce LLM context size and in turn reduce RAG server response latency.
  • Exposed config to split PDFs post chunking. Controlled using APP_NVINGEST_ENABLEPDFSPLITTER environment variable in ingestor-server. Default value is set to True.
  • Added batch-based ingestion which can help manage memory usage of ingestor-server more effectively. Controlled using ENABLE_NV_INGEST_BATCH_MODE and NV_INGEST_FILES_PER_BATCH variables. Default value is True and 100 respectively.
  • Removed extract_options from API level of ingestor-server.
  • Resolved an issue during bulk ingestion, where ingestion job failed if ingestion of a single file fails.

Known Issues

The following are the new known issues in this version:

  • While trying to upload multiple files at the same time, there may be a timeout error Error uploading documents: [Error: aborted] { code: 'ECONNRESET' }. Developers are encouraged to use API's directly for bulk uploading, instead of using the sample rag-frontend. The default timeout is set to 1 hour from UI side, while uploading.
  • In case of failure while uploading files, error messages may not be shown in the user interface of rag-frontend. Developers are encouraged to check the ingestor-server logs for details.

For the full list of known issues, see Known Issues.

Version 2.0.0 (2025-03-18)

This release adds support for multimodal documents using Nvidia Ingest including support for parsing PDFs, Word and PowerPoint documents. It also significantly improves accuracy and perf considerations by refactoring the APIs, architecture as well as adds a new developer friendly UI.

Added

Changed

Known Issues

The following are the new known issues in this version:

  • Optional features reflection, nemoguardrails and image captioning are not available in helm based deployment.
  • Uploading large files with .txt extension may fail during ingestion, we recommend splitting such files into smaller parts, to avoid this issue.

For the full list of known issues, see Known Issues.

Version 1.0.0 (2025-01-15)

This is the initial release of the NVIDIA RAG Blueprint.

All Known Issues

The following are the known issues for RAG Blueprint:

  • The Blueprint responses can have significant latency when using NVIDIA API Catalog cloud hosted models.
  • The accuracy of the pipeline is optimized for certain file types like .pdf, .txt, .docx. The accuracy may be poor for other file types supported by NvIngest, since image captioning is disabled by default.
  • The UI file upload interface has a hard limit of 100 files per upload batch. When selecting more than 100 files, only the first 100 are processed. For bulk uploads beyond this limit, use multiple upload batches or the programmatic API.
  • When updating model configurations in Kubernetes values.yaml (for example, changing from 70B to 8B models), the RAG UI automatically detects and displays the new model configuration from the backend. No container rebuilds are required - simply redeploy the Helm chart with updated values and refresh the UI to see the new model settings in the Settings panel.
  • The NeMo LLM microservice can take 5-6 minutes to start for every deployment.
  • B200 GPUs are not supported for the following advanced features. For these features, use H100 or A100 GPUs instead.
    • Image captioning support for ingested documents
    • NeMo Guardrails for guardrails at input/output
    • VLM based inferencing in RAG
    • PDF extraction with Nemoretriever Parse
  • Sometimes when HTTP cloud NIM endpoints are used from deploy/compose/.env, the nv-ingest-ms-runtime still logs gRPC environment variables. Following log entries can be ignored.
  • If one of the file in a bulk ingestion job is of type svg, which is a unsupported format, the full bulk ingestion job fails.
  • Complicated filter expressions with custom metadata while sending a query, are not supported from the RAG UI.
  • For MIG support, currently the ingestion profile has been scaled down while deploying the chart with MIG slicing This affects the ingestion performance during bulk ingestion, specifically large bulk ingestion jobs might fail.
  • Individual file uploads are limited to a maximum size of 400 MB during ingestion. Files exceeding this limit are rejected and must be split into smaller segments before ingesting.
  • llama-3.3-nemotron-super-49b-v1.5 model provides more verbose responses in non-reasoning mode compared to v1.0. For some queries the LLM model may respond with information not available in given context. Also for out of domain queries the model may provide responses based on it's own knowledge. Developers are strongly advised to tune the prompt for their usecases to avoid these scenarios.
  • The auto selected NIM-LLM profile for llama-3.3-nemotron-super-49b-v1.5 may not work for some GPUs. Follow steps outlined in the appropriate deployment guide to select an optimized profile using NIM_MODEL_PROFILE before deploying.
  • Slow VDB upload is observed in Helm deployments for Elasticsearch.
  • Immediately after document ingestion, there might be a delay before the RAG UI accurately reflects the number of Milvus entities in a collection. Although the count that appears might be temporarily inconsistent, the presence of a document in the RAG UI confirms its successful ingestion.

Related Topics