PhD Research Copilot

An evidence-governed AI research copilot built in Python for scholarly retrieval, document-grounded question answering, reference verification, and reliability-aware LLM workflows.

The repository combines hybrid dense/BM25 retrieval, Sentence Transformers, ChromaDB, claim-specific evidence validation, structured abstention logic, auditable evidence routing, and an experimental EviRoute-RAG prototype. It is a research-oriented engineering artifact, not a production service or a substitute for researcher judgment.

Key principle: Retrieved context is not automatically sufficient to generate an answer. EviRoute-RAG checks whether evidence is relevant, claim-specific, traceable, and internally consistent enough to license a response.

Key capabilities

Area	What the repository demonstrates
Research retrieval	Document ingestion, vector retrieval with ChromaDB, Sentence Transformers embeddings, and BM25 lexical matching
Evidence-grounded QA	Retrieval-augmented answers with source and page references where available
Scholarly discovery	Semantic Scholar, Crossref, and arXiv search workflows
Reference verification	Passage-level hybrid scoring with optional local NLI-style entailment support in the original notebook
EviRoute-RAG	Evidence obligations, RRF ranking fusion, claim-specific support mapping, conflict diagnostics, vetoes, routing, and an evidence ledger
Reliability engineering	Explicit STRONG / BOUNDED / ABSTAIN response states, machine-readable veto reasons, and controlled failure-mode tests

EviRoute-RAG

EviRoute-RAG extends conventional retrieval-augmented generation with an evidence-governance layer. Rather than treating retrieved passages as automatically sufficient, it decomposes a question into evidence obligations, records which passages support each obligation, evaluates evidence quality, and applies hard vetoes before licensing a response.

License state	Meaning
STRONG	Evidence is sufficiently supported, traceable, and free of a hard veto; answer with explicit evidence boundaries.
BOUNDED	Evidence supports only a qualified answer; state the limitations clearly.
ABSTAIN	Evidence is missing, contradictory, insufficiently relevant, or fails provenance/governance checks.

The implementation uses the term Evidence Quality Score, not confidence. It is an engineering diagnostic built from prototype design choices; it is not a calibrated probability and does not prove truth.

Architecture

flowchart LR
    Q[Research question] --> O[Evidence obligations]
    D[Dense semantic retrieval] --> F[RRF fusion]
    B[BM25 lexical retrieval] --> F
    F --> V[Claim-specific support validation]
    V --> C[Conflict graph]
    C --> S[Evidence Quality Score]
    S --> G[Structured veto layer]
    G --> R{Response license}
    R -->|STRONG / BOUNDED / ABSTAIN| L[Auditable evidence ledger]

Loading

How it works

Component	Role
Hybrid retrieval	Combines semantic similarity with exact lexical matching from dense and BM25 retrieval paths.
Reciprocal Rank Fusion	Combines ranked lists without assuming that raw retrieval-score scales are comparable.
Evidence obligations	Turns multi-part questions into explicit support requirements.
Claim-specific validation	Prevents merely topically related passages from being treated as support for a claim.
Conflict graph	Records pairwise contradiction links that can be supplied by an NLI component.
Structured veto layer	Separates evidence quality from permission to generate an answer.
Evidence ledger	Stores provenance, retrieval ranks, RRF scores, support coverage, conflict links, and routing decisions.

The reusable implementation is in src/eviroute.py. Its controlled fixtures model NLI conflict outputs deterministically, which permits reproducible governance tests without claiming live NLI evaluation. See the full engineering description in docs/methodology.md.

Stress tests

The repository includes four controlled, reproducible routing demonstrations. They test not only a successful case but also missing-evidence, unsupported-claim, and direct-conflict cases.

Demonstration	Expected route	Result
Strong evidence	STRONG	Passed
Missing evidence	ABSTAIN + MISSING_EVIDENCE	Passed
Unsupported question	ABSTAIN	Passed
Conflict stress case	ABSTAIN + HIGH_CONFLICT	Passed

The conflict case is intentionally revealing: it has an Evidence Quality Score of 0.976 and complete obligation coverage, but a contradiction edge triggers the HIGH_CONFLICT hard veto. This demonstrates that high retrieval quality alone does not authorize generation. The exact controlled outputs and reproduction command are documented in examples/stress_test_results.md.

Repository structure

phd-research-copilot/
├── docs/
│   └── methodology.md
├── examples/
│   ├── run_stress_tests.py
│   └── stress_test_results.md
├── notebooks/
│   ├── AI_Research_Copilot.ipynb
│   └── EviRoute_RAG_Evidence_Governed_Copilot.ipynb
├── src/
│   ├── __init__.py
│   └── eviroute.py
├── .env.example
├── .gitignore
├── README.md
└── requirements.txt

Installation

The project requires Python 3.10 or later. Create and activate a virtual environment, then install the dependencies.

git clone https://github.com/Haifawaeedd/phd-research-copilot.git
cd phd-research-copilot
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

For LLM-enabled notebook functions, create a local .env file from the example and add your own API key.

cp .env.example .env

OPENAI_API_KEY=your_api_key_here

Running the notebooks and tests

The legacy notebook, notebooks/AI_Research_Copilot.ipynb, contains the broader research-copilot exploration. The primary EviRoute-RAG artifact is notebooks/EviRoute_RAG_Evidence_Governed_Copilot.ipynb. Open it in JupyterLab or VS Code and run the cells from top to bottom.

Run the controlled demonstrations from the repository root:

python examples/run_stress_tests.py

Core technologies

The project uses Python, OpenAI API workflows, ChromaDB, Sentence Transformers, BM25, Gradio, scikit-learn, and document-parsing utilities. The original reference-verification workflow also supports optional transformers and torch dependencies for local NLI-style entailment checks.

Limitations

This is an experimental prototype. The stress tests are controlled demonstrations rather than a large benchmark, the current thresholds are not statistically calibrated decision boundaries, and source independence is approximated by source identity. The conflict interface can consume NLI outputs, but NLI models can misclassify nuanced scientific disagreements. Claim decomposition is intentionally lightweight, and API/model version changes can alter live behavior. Researchers must verify every generated claim and citation against the original source material.

The repository does not claim LangChain, LangGraph, Text-to-SQL, PostgreSQL/PostGIS, production deployment, research novelty, a new algorithm, or state-of-the-art performance.

Security

No API keys, tokens, or notebook outputs are stored in the repository. Keep credentials only in a local .env file, which is excluded from version control. Review source provenance, context, and citations before relying on any generated research output.

Author

Haifaa Owayed
Master of Interdisciplinary Artificial Intelligence (MIAI), University of Ottawa