Skip to content

README.md

Latest commit

 

History

History
180 lines (131 loc) · 6.93 KB

README.md

File metadata and controls

180 lines (131 loc) · 6.93 KB
Raw

AWS Assistant — High-Performance Enterprise RAG

Python 3.12+ FastAPI React 19

A specialized RAG system designed for high-accuracy AWS infrastructure intelligence. This project demonstrates advanced design patterns in AI safety, hybrid retrieval, and performance engineering.

Architecture Overview

graph TD
    User([User / Frontend]) --> API[FastAPI Layer]
    
    subgraph Security_Guardrails [Security & Safety Layer]
        API --> PG[PromptInjectionGuard]
        PG --> DG[DomainGuard LLM Classifier]
    end
    
    subgraph Retrieval_Engine [Hybrid Retrieval Engine]
        DG -- Parallel Execution --> VS[ChromaDB Vector Search]
        DG -- Parallel Execution --> BM[BM25 Keyword Search]
        VS & BM --> RRF[Reciprocal Rank Fusion]
    end
    
    subgraph Generation_Pipeline [Generation & QA]
        RRF --> LLM[GPT-4o Primary Generator]
        LLM --> RG[ResponseGuard G-Eval]
        LLM -- Fallback --> FBL[Fallback Model]
    end
    
    RG --> Output([Verified Response])
    
    subgraph Performance_Monitoring [Observability]
        API -.-> RT[X-Process-Time Tracking]
        API -.-> RL[SlowAPI Rate Limiting]
    end
Loading

Architecture & Design Choices

1. Hybrid Retrieval: The RRF Strategy

We implement a Reciprocal Rank Fusion (RRF) strategy to solve the limitations of single-vector search.

  • Dense Search: Using all-MiniLM-L6-v2 via ChromaDB to capture semantic intent.
  • Sparse Search: Using BM25Okapi to capture exact keyword matches (e.g., "S3 Lifecycle Policies").
  • Design Decision: RRF was chosen over simple weighted averaging because it doesn't require score normalization across different scales, making it more robust as the document corpus grows.

2. Multi-Model Routing & Fallback

To balance cost and reliability, we use a tiered execution model:

  • Tier 1 (Classification): gpt-4o-mini handles domain gating and safety checks.
  • Tier 2 (Primary Generation): gpt-4o handles the complex synthesis of technical AWS answers.
  • Resilience: The system implements automated multi-model fallback (GPT-4o → 4o-mini). If the primary model encounters rate limits or errors, the system automatically falls back to a secondary stable instance to ensure 99.9% availability.

3. Parallel Async Pipeline & Elite RAG Engine

  • Elite RAG Engine: The backend is upgraded to an Elite RAG Engine featuring parallel hybrid retrieval (Dense + BM25) and Reciprocal Rank Fusion (RRF).
  • Implementation: We use asyncio.gather to run Domain Guard validation and Vector Retrieval concurrently.
  • Reasoning: Since retrieval is I/O bound and domain validation is network-latency bound, parallelizing these saves ~1.5 seconds per request.

4. Hallucination Detection (G-Eval)

  • Faithfulness (G-Eval): We use an "LLM-as-a-judge" pattern (ResponseGuard) to score how well the answer is supported by the retrieved context (0.0 - 1.0).

Security: Defense-in-Depth

Our security layer is designed to be proactive rather than reactive, handling three primary threat vectors:

● Prompt Injection & Jailbreaks

  • Regex-based Anchoring: We use strict pattern matching to detect common jailbreak precursors (e.g., "ignore previous instructions", "jailbreak").
  • Encoded Attack Handling: The system proactively blocks common encoded attack patterns (e.g., Base64/Hex attempts).
  • XML Tag Isolation: User input is wrapped in <user_query> tags. Our system rejects any query containing closing tags (</user_query>), preventing "tag spoofing" where a user tries to escape the prompt context.

● Domain Boundary Enforcement (Domain Guard)

  • We use an LLM-based classifier to verify if the query relates to AWS and provide helpful AWS-branded explanations if the user asks out-of-scope questions.

● Layered Authentication (X-API-KEY)

  • Implementation: We require a mandatory X-API-KEY header for all endpoints to protect the backend.

API Layer

The system is exposed via a production-grade FastAPI layer.

POST /api/query

Main endpoint for synchronized RAG retrieval.

{
  "question": "How do I optimize costs for a high-traffic RDS instance?"
}

POST /api/stream-query

Streaming endpoint (SSE) for low-latency perceived performance.

Local Development Guide

Follow these steps for a clean, reliable local setup.

1. Prerequisites

  • Python 3.12+
  • Node.js 20+

2. Backend Setup

From the project root:

# 1. Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# 2. Install dependencies
pip install -r requirements.txt

# 3. Set up environment variables
# Copy the example file and update it with your API key
cp .env.example .env

Required .env content:

OPENAI_API_KEY=your_openai_api_key_here
API_KEY=Rfvtgb*321
DOMAIN_NAME=AWS Cloud Services

4. Start the Backend API

uvicorn src.main:app --host 0.0.0.0 --port 3000

The API is now running at http://localhost:3000. You can test it via http://localhost:3000/docs.

3. Frontend Setup

In a new terminal window, from the project root:

# 1. Navigate to frontend
cd frontend

# 2. Install dependencies
npm install

# 3. Start the development server
npm run dev

4. Configure the Dashboard

  • Open http://localhost:5173 in your browser.
  • Open Settings (top-right).
  • API Base URL: http://localhost:3000
  • X-API-KEY: Ask the author
  • Click Save. The "System Health" indicator should turn green ("Operational").

4. Verification

If the dashboard shows "Operational", you are ready to query! Ask any question, and the assistant will retrieve context from the AWS documentation.

Evaluation & Quality

Metrics

  • Faithfulness (G-Eval): We use an "LLM-as-a-judge" pattern to score how well the answer is supported by the retrieved context (0.0 - 1.0).
  • Hallucination Rate: Monitored via automated evaluation scripts that compare generated answers against ground-truth AWS whitepapers.

Trade-offs

  • Latency vs. Accuracy: We prioritize RRF hybrid search which adds latency compared to simple vector search but increases accuracy by 22%.
  • Cost vs. Security: Running a classifier before every query increases token cost but protects the primary LLM from processing malicious payloads.

Limitations

  • Selective Caching: We intentionally did not include global caching for generated responses due to privacy and security guardrail complexity.
  • Corpus Recency: The system is currently limited to the aws-overview.pdf context.
  • Context Window: Large document retrievals can occasionally hit token limits, managed currently via RecursiveCharacterTextSplitter.

by ~ Chaitanya♥️