architecture.md

Architecture

RAG Assistant is a modular Retrieval-Augmented Generation engine. Each stage is a small, replaceable component; the RAGPipeline wires them together and is itself embeddable inside a larger LangChain application.

                      ┌──────────────┐
   corpus  ─────────▶ │  Ingestion   │  load + sentence-aware chunking
                      └──────┬───────┘
                             ▼
                      ┌──────────────┐
                      │  Embeddings  │  sentence-transformers bi-encoder
                      └──────┬───────┘
                             ▼
        ┌────────────────────┴────────────────────┐
        ▼                                          ▼
 ┌─────────────┐                            ┌─────────────┐
 │ Dense / FAISS│                           │ Sparse / BM25│
 └──────┬──────┘                            └──────┬──────┘
        └───────────────┬──────────────────────────┘
                        ▼
                 ┌──────────────┐
                 │ Hybrid Fusion│  Reciprocal Rank Fusion / weighted
                 └──────┬───────┘
                        ▼
                 ┌──────────────┐
                 │ Cross-Encoder│  re-rank shortlist for precision
                 │  Re-ranker   │
                 └──────┬───────┘
                        ▼
                 ┌──────────────┐
                 │  Generation  │  vLLM (Llama-3-8B + QLoRA adapter)
                 └──────┬───────┘
                        ▼
                  grounded answer + cited sources

Stages

Ingestion (ingestion/)

load_corpus reads .txt, .md and .jsonl files and chunk_text splits them into overlapping, sentence-aware windows. Overlap preserves answers that straddle chunk boundaries. Each chunk becomes a Document with a stable content hash id.

Embeddings (embeddings/)

Encoder wraps a sentence-transformers bi-encoder, batching inputs and L2-normalizing vectors so inner-product search equals cosine similarity. The model loads lazily on first use.

Retrieval (retrieval/)

• Dense (dense.py): a FAISS flat inner-product index over chunk embeddings, persisted to disk alongside its document store.
• Sparse (sparse.py): Okapi BM25 over tokenized chunks, matching rare terms embeddings miss.
• Hybrid (hybrid.py): fuses the two candidate lists with Reciprocal Rank Fusion (default) or a min-max weighted sum.
• Re-ranker (reranker.py): a cross-encoder scores (query, document) pairs jointly and keeps the top n as generation context.

Generation (generation/)

build_prompt grounds the question in the retrieved context with a faithfulness -oriented system prompt. VLLMClient calls a self-hosted vLLM server over its OpenAI-compatible API; TransformersClient is an offline fallback.

Configuration

All knobs live in config/config.yaml and are overridable by environment variables of the form RAG_<SECTION>__<KEY> (see .env.example).

Persistence

RAGPipeline.save / .load persist the FAISS index, the BM25 state and the document store under index/, so a corpus is processed once and queried many times.