Knowledge Agent

Knowledge Agent provides a personal research knowledge base. You chat with an AI agent that can search arXiv, ingest papers into a local vector store, and answer questions over the papers you've saved. Architecturally, the agent's abilities are exposed as MCP tools served by independent subprocesses. The LLM drives an agentic tool-call loop to decide when to search/save/recall, and saved papers are embedded locally to facilitate semantic search and retrieval-augmented answers.

Architecture

Data flow. The frontend calls the backend only (never the LLM directly). For chat, the backend's process_query() runs an agentic loop: it sends the conversation to the LLM via OpenRouter, the LLM requests tools, the MCP client routes each call over stdio to the owning server, results are fed back, and the loop repeats until the LLM returns a final answer. arxiv_server hits the public arXiv API; ingest_server writes embeddings to ChromaDB and metadata to SQLite; search_server reads ChromaDB by semantic similarity. The /api/library endpoint reads SQLite directly (no LLM needed) so the sidebar is fast and works regardless of agent state.

Tech stack

Component	Technology
Frontend	React + Vite
Backend	FastAPI + Uvicorn
LLM access	OpenRouter via the openai SDK
Model	google/gemini-2.5-flash
Tools	MCP (Model Context Protocol), stdio transport
Embeddings	all-MiniLM-L6-v2 (ChromaDB default, ONNX)
Metadata	SQLite
Package manager	uv

Setup

Prerequisites

• Python 3.12+ (required by pyproject.toml)
• Node.js 18+ (for the Vite frontend)
• uv for Python dependency management
• An OpenRouter API key

Clone and install

git clone <your-repo-url> knowledge-agent
cd knowledge-agent

# Backend: uv reads pyproject.toml + uv.lock and builds the venv
uv sync

# Frontend
cd frontend && npm install && cd ..

Configure environment

cp .env.example .env
# then edit .env and set your key:
# OPENROUTER_API_KEY=sk-or-...

Run the backend

uv run uvicorn backend.main:app --reload
# serves http://localhost:8000

Run the frontend

cd frontend && npm run dev
# serves http://localhost:5173

Open http://localhost:5173 in the browser.

Usage

Try these queries in order to see all features:

1. Search for papers:

   search for 2 papers on LLM agents

   The agent calls search_arxiv and returns PaperCards (title, authors, date, summary with show more).

2. Save a paper: click Save to Library on one of the cards. The agent calls ingest_paper (embeds the summary into ChromaDB + stores metadata in SQLite); the card shows Saved ✓ and the paper appears in the Library sidebar.

3. Ask about any saved papers:

   what do my saved papers say about agents

   The agent runs semantic_search over the library and answers in prose, grounded in the papers that are saved (this is the RAG step).

4. Browse / revisit the library: the right sidebar lists saved papers; click one to ask the chat:

   tell me about paper <id>

Key concepts

MCP (Model Context Protocol). A standard for exposing "tools" to an LLM. Here each capability (arXiv search, ingest, semantic search) is a small MCP server running as its own subprocess, speaking JSON-RPC over stdio. This keeps concerns and dependencies isolated and makes the same tools reusable by any MCP client.

The agentic loop. The LLM is a decision-maker, not a doer. process_query() sends the conversation to the model; if it requests tools, the backend executes them, appends the results, and sends everything back, repeating until the model produces a final text answer. The model decides which tools to call and when.

RAG (Retrieval-Augmented Generation). Instead of relying on the model's memory, relevant text is retrieved and placed into the prompt. Here, saved papers are embedded into ChromaDB; semantic_search finds the closest ones to the question by meaning, and the LLM composes an answer grounded in that retrieved text, so answers reflect the library and cite real papers.

Project structure

knowledge-agent/
├── CLAUDE.md                  # Project spec & conventions for AI-assisted work
├── README.md                  
├── pyproject.toml             # Python project + dependencies (uv)
├── uv.lock                    # Pinned, reproducible dependency versions
├── .python-version            # Python interpreter pin (3.12)
├── .env.example               # Template for required env vars
├── .env                       # env vars
├── .gitignore
│
├── backend/
│   ├── config.py              # Single source of config: paths, model, keys, CORS
│   ├── main.py                # FastAPI app: /api routes + MCP lifespan wiring
│   └── mcp_client.py          # Spawns MCP servers, tool discovery, agentic loop
│
├── mcp_servers/
│   ├── arxiv_server.py        # MCP tools: search_arxiv, fetch_paper (arXiv API)
│   ├── ingest_server.py       # MCP tools: ingest_paper, list_library (ChromaDB + SQLite)
│   └── search_server.py       # MCP tool: semantic_search (ChromaDB)
│
├── frontend/
│   ├── index.html             # Vite entry HTML
│   ├── package.json           # Frontend deps & scripts
│   ├── vite.config.js         # Vite configuration
│   └── src/
│       ├── main.jsx           # React entry point
│       ├── App.jsx            # Root layout; wires ChatPanel + Library
│       ├── App.css            # Dark-theme styling
│       ├── index.css          # Base styles
│       └── components/
│           ├── ChatPanel.jsx  # Chat UI; calls /api/chat, renders PaperCards
│           ├── PaperCard.jsx  # Single paper: summary toggle, Save, PDF link
│           └── Library.jsx    # Sidebar of saved papers
│
└── data/                      # Local persistence 
    ├── chroma/                # ChromaDB vector store
    └── papers.db              # SQLite metadata