This blueprint is for developers who want a quick start to set up a RAG solution with a path-to-production with NVIDIA NIM.
Note This blueprint runs in NVIDIA AI Workbench. It's a free, lightweight developer platform that you can run on your own systems to get up and running with complex AI applications and workloads in a short amount of time.
You may want to fork this repository into your own account before proceeding. Otherwise you won't be able to fully push any changes you make because this NVIDIA-owned repository is read-only.
Navigating the README: Project Overview | Get Started | License | Terms of Use
Other Resources: ⬇️ Download AI Workbench | 📖 User Guide |📂 Other Projects | 🚨 User Forum
This blueprint serves as a reference solution for a foundational Retrieval Augmented Generation (RAG) pipeline. One of the key use cases in Generative AI is enabling users to ask questions and receive answers based on their enterprise data corpus. This blueprint demonstrates how to set up a RAG solution that uses NVIDIA NIM and GPU-accelerated components. By default, this blueprint leverages locally-deployed NVIDIA NIM microservices to meet specific data governance and latency requirements. However, you can replace these models with your NVIDIA-hosted models available in the NVIDIA API Catalog.
Ensure you have satisfied the prerequisites for this Blueprint (details).
-
Open NVIDIA AI Workbench. Select a Location to work in.
-
Clone the project with URL: https://github.com/NVIDIA-AI-Blueprints/rag
-
On the Project Dashboard, resolve the yellow unconfigured secrets warning by inputting your
NGC_API_KEY. -
Select
ingest,rag,vectordb, andlocalcompose profiles from the dropdown under the Compose section.- Note:
observabilityandguardrailsare optional profiles you may enable.
- Note:
-
Select Start. The compose services may take several minutes to pull and build.
-
When all compose services are ready, access the frontend on the IP address, eg.
http://<ip_addr>:8090. -
You can now interact with the RAG Chatbot through its browser interface.
-
Open NVIDIA AI Workbench. Select a Location to work in.
-
Clone the project with URL: https://github.com/NVIDIA-AI-Blueprints/rag
-
On the Project Dashboard, resolve the yellow unconfigured secrets warning by inputting your
NGC_API_KEY. -
Under File Browser, locate the
variables.envfile. -
From the hamburger menu, select Edit. Make the following edits:
- Comment out the variables for
on-prem NIMs - Uncomment the variables for using
cloud NIMs - Save your changes
- Comment out the variables for
-
On the Project Dashboard, select
ingest,rag, andvectordbcompose profiles under the Compose section.- Note:
observabilityandguardrailsare optional profiles you may enable. Do not selectlocal.
- Note:
-
Select Start. The compose services may take several minutes to pull and build.
-
When the compose services are ready, access the frontend on the IP address, eg.
http://<ip_addr>:8090. -
You can now interact with the RAG Chatbot through its browser interface.
This NVIDIA AI BLUEPRINT is licensed under the Apache License, Version 2.0. This project will download and install additional third-party open source software projects and containers. Review the license terms of these open source projects before use.
Use of the models in this blueprint is governed by the NVIDIA AI Foundation Models Community License.
This blueprint is governed by the NVIDIA Agreements | Enterprise Software | NVIDIA Software License Agreement and the NVIDIA Agreements | Enterprise Software | Product Specific Terms for AI Product. The models are governed by the NVIDIA Agreements | Enterprise Software | NVIDIA Community Model License and the NVIDIA RAG dataset which is governed by the NVIDIA Asset License Agreement.
The following models that are built with Llama are governed by the Llama 3.2 Community License Agreement: nvidia/nemotron-3-super-120b-a12b, nvidia/llama-nemotron-embed-1b-v2, and nvidia/llama-nemotron-rerank-1b-v2.