Contract Compliance Review Agent

An agent that reviews a contract (NDA, vendor agreement, or DPA) against a company's internal policy thresholds and a database of previously-approved clause language, then produces a clause-by-clause compliance report with risk levels and citations.

Built with the raw Anthropic SDK — no agent framework — so the orchestration loop, tool routing, and self-check logic are all visible and readable in agent/orchestrator.py.

Why this project

Most "agent" demos are a chatbot with retrieval bolted on. This one is built around a real workflow with actual decision points: the agent has to route between two different knowledge sources (unstructured policy text vs. a structured precedent database), apply a deterministic risk-scoring table rather than freelancing severity judgments, and re-check its own high-risk/missing findings before finalizing them — a concrete, demonstrable self-correction step rather than a single pass through an LLM.

It's also been genuinely exercised, not just demoed once: four sample contracts across all three supported document types, and four real bugs found and fixed along the way (see Bugs found and fixed).

Architecture

User provides contract text
        │
        ▼
[classify_document]  ── LLM sub-call → doc_type + confidence
        │
        ▼
[get_required_clauses] ── DB lookup → which clauses apply to this doc_type
        │
        ▼
[extract_clauses]  ── LLM sub-call → clause text per required clause
        │
        ▼
For each clause:
    [search_policy]      ── keyword lookup over policy markdown (RAG-lite)
    [query_precedents]   ── SQL lookup over approved clause language
    [lookup_risk_rule]   ── SQL lookup → deterministic risk level
    (self-check re-read for high-risk / missing findings)
    [record_finding]     ── writes to findings table
        │
        ▼
[finalize_report] ── rejected until every required clause has a finding;
                      compiles findings into a markdown report

The main loop (ComplianceAgent.review()) is a straightforward call-model → execute-tool-calls → feed-results-back cycle. Nothing here is hidden inside a framework; every tool call the model makes is logged to agent.transcript and can be printed with --transcript.

Data layer vs. reasoning layer

• policies/*.md — unstructured policy text, searched via simple header-keyword matching (db_tools.search_policy). Three files is too small to justify a vector store; if the policy corpus grew to dozens of documents, this is the piece you'd swap for embeddings.
• precedent_db/precedent.db — structured SQLite DB: clause types, approved language precedents, and — critically — a risk_rules table the agent is instructed to always consult rather than inventing a risk level itself. This is what keeps repeated runs consistent.
• LLM calls — used only for the two genuinely open-ended sub-tasks: classifying a document and extracting clause text from unstructured contract prose. Everything else (risk tiering, precedent lookup) is plain SQL.

Setup

pip install -r requirements.txt
export ANTHROPIC_API_KEY=sk-...

# Build the precedent DB (only needed if you edit schema.sql / seed_data.sql)
cd precedent_db && python3 build_db.py && cd ..

Run it

python main.py sample_contracts/acme_vendor_agreement.txt

Add --transcript to see every tool call the agent made, in order — useful for demoing the decision-making, not just the final report:

python main.py sample_contracts/acme_vendor_agreement.txt --transcript

Reports are also saved to reports/<filename>_report.md.

Sample contracts

Four sample contracts are included, deliberately covering both flawed and clean documents across all three supported types:

File	Type	Expected result
acme_vendor_agreement.txt	vendor agreement	Flawed — liability cap below threshold, missing indemnification, no breach notification, termination notice too long. 3 high / 3 medium / 3 low risk findings.
acme_vendor_agreement_compliant.txt	vendor agreement	Fully compliant — every clause sits at or within policy thresholds. 9/9 compliant, 0 flags.
solara_nda.txt	NDA	Flawed — perpetual confidentiality term, a prohibited non-solicitation clause, a missing exclusion, non-standard governing law. 2 high / 2 medium / 1 low.
vertex_dpa.txt	DPA	Flawed — generic security language, breach notification window too long, missing sub-processor and audit clauses, no liability carve-out. 3 high / 3 medium.

Running the flawed and compliant vendor agreements back to back is the clearest demo: same code, same policies, opposite outcomes — proof the agent is actually reasoning against the rules rather than pattern-matching to "always flag something."

Bugs found and fixed

Built and debugged against the live API, not just written and assumed to work. Four distinct issues surfaced across real runs, spanning two different categories — code/orchestration bugs and a data-completeness gap in the policy corpus itself:

1. Thinking-block parsing. llm_tools.py originally assumed a model response's first content block was always plain text. When extended thinking is involved, it isn't — resp.content[0] can be a thinking block, silently breaking classify_document/extract_clauses. Fixed by filtering resp.content for the actual text block instead of assuming its position.
2. Premature finalization. The agent occasionally called finalize_report before recording a finding for every required clause, producing an empty or partial report with no error. Fixed by making finalize_report check completeness against the database independently — it's rejected with a specific "here's what's still missing" message until every required clause actually has a recorded finding.
3. Silent truncation. A response cut off mid-turn (stop_reason == "max_tokens") could leave a dangling, unresolved tool call. Fixed by raising max_tokens and treating truncation as a loud failure rather than a quiet, misleading "success."
4. Missing policy section. dpa_policy.md never had a dedicated "Governing Law" section, even though governing_law is a required clause for every document type per the precedent DB. search_policy correctly reported no match, and the agent worked around it by citing the NDA policy's rule instead — reasonable behavior, but the wrong citation ended up in a DPA report. This wasn't a code bug; it was a real gap in the policy corpus itself, only caught by actually testing the DPA path. Fixed by adding the missing section.

Project structure

compliance-agent/
├── policies/                    # policy markdown (RAG-lite source)
├── precedent_db/                # schema, seed data, build script, DB file
├── agent/
│   ├── db_tools.py               # deterministic SQL / file lookups
│   ├── llm_tools.py               # classify_document, extract_clauses
│   ├── tool_specs.py             # Anthropic tool-use JSON schemas
│   └── orchestrator.py           # the agent loop
├── sample_contracts/             # test documents (flawed + compliant, all 3 doc types)
├── reports/                      # generated output (gitignored)
└── main.py                       # CLI entrypoint

What would come next

• Swap search_policy's keyword matching for embeddings if the policy corpus grows past a handful of documents.
• A thin Streamlit wrapper for a nicer live demo (optional — the CLI + saved report is the core deliverable).
• A lint/consistency check that verifies every clause type in the precedent DB has a matching policy section, to catch gaps like bug #4 automatically instead of by manual testing.