Vector

A Python library for PDF ingestion, semantic search, and visual grounding. Convert PDFs to structured documents, chunk them intelligently, embed with sentence-transformers, and retrieve relevant passages with bounding-box metadata for highlighting source text.

Repository: https://github.com/dkylewillis/vector.git

Features

PDF → DoclingDocument conversion (text-native; no in-memory page image generation)
On-demand page rendering from stored PDFs via pypdfium2 for visual grounding
Hybrid chunking (structure-aware + token-limit-aware)
Figure extraction: PictureItem and TableItem elements detected during chunking, cropped from stored PDFs as PNGs, and associated with their nearby text chunks
ChromaDB vector store with cosine similarity search
Context window: fetch N chunks before/after each hit
Multi-document search with per-document filtering
Visual grounding: normalized bounding boxes on every chunk for PDF highlighting
Multi-page chunk tracking: page_range field records all pages a chunk spans
Ingestion status tracking (pending → complete / failed) with repair support
Per-stage progress signals during ingest (converting → storing → chunking → embedding → extracting figures)
Machine-typed error codes on all failures for reliable agent branching
Document manifest for fast listing without loading full documents
Document renaming without re-embedding
Force re-ingest (--force) for updated documents
JSON-first CLI designed for AI agent consumption

Data layout

Each collection is fully self-contained:

vector_data/
  vector/                        ← default collection
    manifest.json                 ← metadata + ingestion status + tags + figures for every document
    documents/
      <hash>/
        source.pdf                ← original PDF copy (used for on-demand rendering and repair)
        figures/
          pic_000.png             ← extracted picture PNGs (created with --extract-figures)
          tbl_000.png             ← extracted table PNGs
    embeddings/                   ← ChromaDB persist directory
      chroma.sqlite3
      <uuid>/
  legal/                          ← python cli.py --collection legal ingest ...
    manifest.json
    documents/
    embeddings/

Modules

`models.py`

Class	Purpose
`BoundingBox`	Normalized (0–1) page region, top-left origin
`FigureRef`	Lightweight figure reference stored on each chunk (ref, kind, page, bbox, caption)
`FigureRecord`	Full stored metadata for an extracted figure image (id, paths, nearby chunk indices)
`Chunk`	Text passage with headings, bboxes, figure refs, page range, and document provenance
`DocumentRecord`	Manifest entry: metadata + `status` + `has_figures` + `tags` dict + `chunk_count` (cached from ingest)
`QueryResult`	Semantic search hit + surrounding context window

`converter.py`

Converter()
  .convert(source: str) -> DoclingDocument
  .convert_page_range(source: str, start_page: int, end_page: int) -> DoclingDocument
  .convert_in_page_chunks(source: str, chunk_size: int) -> Generator[(start, end, DoclingDocument)]
  .page_count(source: str) -> int                        # static; fast pypdfium2 read, no ML

Use convert_page_range or convert_in_page_chunks for large PDFs with memory-spike pages. Each page range is converted independently so peak memory is bounded to chunk_size pages.

`rendering.py`

render_page(pdf_path: str, page_number: int, scale: float = 2.0) -> PIL.Image
crop_figure(pdf_path: str, bbox: BoundingBox, scale: float = 2.0) -> PIL.Image

render_page opens a single page from a stored PDF on demand — no memory overhead at ingest time. crop_figure renders a page and crops to a normalized bounding box — used during figure extraction.

`chunker.py`

Chunker(dl_doc: DoclingDocument, tokenizer="sentence-transformers/all-MiniLM-L6-v2")
  .chunk() -> List[Chunk]

`document_store.py`

DocumentStore(base_path="./vector_data/vector")   # base_path = collection root
  .create(dl_doc, source_pdf_path, page_count=None) -> str  # copies PDF, writes status=pending
  .set_status(file_hash, status)                          # "pending" | "complete" | "failed"
  .update_chunk_count(file_hash, count: int)              # cache embedded chunk count in manifest
  .get_pdf_path(file_hash)          -> Path               # path to stored PDF copy
  .exists(file_hash)                -> bool               # True only if status=complete
  .delete(file_hash)                                      # removes documents/<hash>/ dir + manifest entry
  .list()                           -> List[DocumentRecord]
  .list_incomplete()                -> List[DocumentRecord]  # status != complete
  .rename(file_hash, new_name)                            # update document_name in manifest
  .update_tags(file_hash, tags: dict)                     # merge tags into manifest entry
  .remove_tags(file_hash, keys: List[str])                # remove specified tag keys
  .save_figures(file_hash, figures, images=None)          # save figure PNGs + embed metadata in manifest
  .get_figures(file_hash)           -> List[FigureRecord]  # load figure records from manifest
  .get_figure_image_path(file_hash, figure_id) -> Path    # path to a specific figure PNG
  .delete_figures(file_hash)                               # remove figures/ subdir for a document

`vector_store.py`

VectorStore(collection_name="vector", persist_directory="./vector_data/vector/embeddings", embedding_model=...)
  .create(chunks, tags=None)        # tags stored as tag_<key> metadata on every chunk
  .query(query_text, top_k=5, file_hash=None, window=0, tags=None) -> List[QueryResult]
  .list_documents()                 -> List[dict]         # unique docs with chunk counts (scans all chunks)
  .has_chunks(file_hash)            -> bool               # fast existence check (limit=1, no full scan)
  .remove_tag_metadata(file_hash, tag_keys: List[str])   # remove tag_ fields from all chunks
  .set_tag_metadata(file_hash, tags: Dict[str, str])     # merge tag_ fields into all chunk metadata
  .delete(file_hash)                                      # remove all chunks for a document
  .update(chunks, tags=None)        # delete then re-add

file_hash in query() accepts None (all docs), a single hash string, or a list of hash strings. tags in query() accepts dict[str, str | list[str]] — all keys must match (AND across keys); list values use OR ($in) within a key.

Python API Usage

from converter import Converter
from rendering import render_page
from chunker import Chunker
from document_store import DocumentStore
from vector_store import VectorStore

# Collection paths — both rooted under the collection directory
COLLECTION      = "vector"
COLLECTION_ROOT = f"./vector_data/{COLLECTION}"
EMBEDDINGS      = f"{COLLECTION_ROOT}/embeddings"

# --- Ingest (whole document) ---
converter = Converter()
dl_doc = converter.convert("report.pdf")

doc_store = DocumentStore(COLLECTION_ROOT)
file_hash = doc_store.create(dl_doc, source_pdf_path="report.pdf")  # status=pending

chunks = Chunker(dl_doc).chunk()

vs = VectorStore(collection_name=COLLECTION, persist_directory=EMBEDDINGS)
vs.create(chunks, tags={"type": "report", "author": "ACME"})
doc_store.update_tags(file_hash, {"type": "report", "author": "ACME"})
doc_store.set_status(file_hash, "complete")
doc_store.update_chunk_count(file_hash, len(chunks))

# --- Ingest (chunked, for large/heavy PDFs) ---
converter = Converter()
total_pages = Converter.page_count("report.pdf")

file_hash = doc_store.create(
    next(doc for _, _, doc in converter.convert_in_page_chunks("report.pdf", chunk_size=50)),
    source_pdf_path="report.pdf",
    page_count=total_pages,
)
tags = {"type": "report"}
chunk_offset = 0
for start, end, dl_doc in converter.convert_in_page_chunks("report.pdf", chunk_size=50):
    range_chunks = Chunker(dl_doc).chunk()
    for chunk in range_chunks:
        chunk.index = chunk_offset
        chunk.id = f"{file_hash}_{chunk_offset}"
        chunk_offset += 1
    vs.create(range_chunks, tags=tags)
doc_store.update_tags(file_hash, tags)
doc_store.set_status(file_hash, "complete")
doc_store.update_chunk_count(file_hash, chunk_offset)

# --- Query (all documents in collection) ---
results = vs.query("transformer architecture", top_k=5, window=1)

# --- Query (specific documents) ---
results = vs.query("transformer architecture", file_hash="11465328351749295394")
results = vs.query("transformer architecture", file_hash=["hash_a", "hash_b"])

# --- Inspect results ---
for r in results:
    print(r.chunk.headings, r.chunk.page_number)
    print([c.index for c in r.context])   # surrounding chunks

# --- Visual grounding ---
top = results[0].chunk
pdf_path = doc_store.get_pdf_path(top.file_hash)
for bbox in top.bboxes:
    img = render_page(str(pdf_path), bbox.page_no)
    # bbox.l / .r / .t / .b are normalized 0-1; multiply by img.width / img.height

CLI

The CLI is the primary interface — all output is machine-readable JSON printed to stdout (one JSON object per line). Long-running commands emit progress lines before the final result. Errors go to stderr as {"error": "...", "error_code": "..."} with a non-zero exit code.

Command summary

Command	Description
`ingest`	Convert a PDF, store it, chunk it, and embed it
`query`	Semantic search across ingested documents
`ask`	Ask a natural-language question (query expansion + search + AI answer)
`list`	List ingested documents
`status`	Show collection health
`collections`	List all collections in the data directory
`repair`	Find and optionally fix incomplete ingestions
`tag`	Set, update, or remove tags on a document
`tags`	List all tag keys and their distinct values across the collection
`figures`	List or export extracted figures for a document
`rename`	Rename a document in the manifest and vector store
`delete`	Remove a document from all stores

Global flags

Every command accepts these two flags to select which collection to operate on:

python cli.py [--data-dir DIR] [--collection NAME] <command> [command-flags]

Flag	Default	Env override	Description
`--data-dir DIR`	`./vector_data`	`VECTOR_DATA_DIR`	Root directory containing all collections
`--collection NAME`	`vector`	—	Name of the collection to operate on. Data is stored at `<data-dir>/<collection>/`

How collections work

Collections are created implicitly — there is no "create collection" command. The first time you run any command targeting a collection name, its directory structure is created automatically:

# This creates vector_data/legal/doc_store/ and vector_data/legal/chroma/ on the fly
python cli.py --collection legal ingest data/contract.pdf

Every subsequent command targeting that collection uses the same --collection flag:

python cli.py --collection legal query "termination clause"
python cli.py --collection legal list
python cli.py --collection legal status

If you omit --collection, the default collection vector is used. Use python cli.py collections to discover all existing collections.

`ingest`

Convert a PDF, store it, chunk it, and embed it into the vector store.

python cli.py ingest <file> [--chunk-size N] [--force] [--extract-figures]

Flag	Default	Description
`<file>`	(required)	Path to the PDF file to ingest
`--chunk-size N`	`25`	Pages per conversion pass. Bounds peak memory for large PDFs. Set to `0` to convert the whole document at once
`--force`	off	Delete existing data and re-ingest, even if the document was already ingested
`--extract-figures`	off	Extract figures (pictures and tables) as PNGs. Figures are cropped from the stored PDF using bounding boxes detected during chunking

Behavior:

The original PDF is copied into the doc store for later rendering and repair.
Re-ingesting the same file (matched by content hash) returns already_ingested immediately — no duplicate work. Use --force to override.
In chunked mode (default), the PDF is converted in 25-page passes. Each pass is converted, chunked, and embedded independently so peak memory stays bounded.
When --extract-figures is enabled, PictureItem and TableItem elements are detected during chunking and their images are cropped from the stored PDF and saved as PNGs.

Progress output — one JSON line per stage before the final result. All progress lines include "type": "progress" so agents can distinguish them from the final result line:

{"type": "progress", "status": "converting", "file": "data/report.pdf", "page_range": "1-25", "range_index": 1, "total_ranges": 8, "total_pages": 195}
{"type": "progress", "status": "storing",    "file_hash": "...", "page_count": 195}
{"type": "progress", "status": "converting", "file": "data/report.pdf", "page_range": "26-50", "range_index": 2, "total_ranges": 8}
{"type": "progress", "status": "chunking",   "page_range": "26-50", "range_index": 2, "total_ranges": 8}
{"type": "progress", "status": "embedding",  "page_range": "26-50", "range_index": 2, "total_ranges": 8, "chunk_count": 38, "chunks_embedded_so_far": 41}

Final result:

{"status": "ingested", "collection": "vector", "file_hash": "11465328351749295394", "document_name": "report", "filename": "report.pdf", "page_count": 195, "chunk_count": 312}

Examples:

# Default chunked mode (25 pages per pass)
python cli.py ingest data/report.pdf

# Larger chunks (50 pages per pass)
python cli.py ingest data/report.pdf --chunk-size 50

# Convert the whole document at once (no chunking)
python cli.py ingest data/report.pdf --chunk-size 0

# Ingest into a different collection
python cli.py --collection legal ingest data/contract.pdf

`query`

Semantic search across ingested documents. Returns the top matching chunks with optional surrounding context.

python cli.py query <text> [--top-k N] [--window N] [--name SUBSTR] [--file-hash HASH ...]

Flag	Default	Description
`<text>`	(required)	Natural-language query string
`--top-k N`	`5`	Maximum number of results to return
`--window N`	`0`	Number of adjacent chunks to include before and after each hit (for context)
`--name SUBSTR`	—	Restrict to documents whose name or filename contains this substring (case-insensitive). Resolved to `file_hash` internally
`--file-hash HASH`	all docs	Restrict to a specific document by hash. Repeat the flag to search across multiple documents

Result structure:

{
  "query": "What are the main AI models?",
  "top_k": 3,
  "window": 1,
  "result_count": 3,
  "results": [
    {
      "chunk": {
        "id": "11465328351749295394_10",
        "index": 10,
        "page_number": 3,
        "headings": ["3.2 AI models"],
        "text": "3.2 AI models\nAs part of Docling...",
        "bboxes": [{"l": 0.176, "r": 0.823, "t": 0.488, "b": 0.582, "page_no": 3}],
        "file_hash": "11465328351749295394",
        "document_name": "report",
        "file_extension": ".pdf"
      },
      "context": [
        {"index": 9, "headings": ["3.1 PDF backends"], "...": "..."},
        {"index": 10, "headings": ["3.2 AI models"], "...": "..."},
        {"index": 11, "headings": ["Layout Analysis Model"], "...": "..."}
      ]
    }
  ]
}

Examples:

# Basic search
python cli.py query "What are the main AI models?" --top-k 3

# Search with surrounding context (1 chunk before + after each hit)
python cli.py query "transformer architecture" --top-k 5 --window 1

# Restrict by document name (case-insensitive substring match)
python cli.py query "front setbacks" --name "coweta"

# Restrict by exact file hash
python cli.py query "tables" --file-hash 11465328351749295394

# Search across two specific documents
python cli.py query "tables" --file-hash 11465328351749295394 --file-hash 99887766

`ask`

Ask a natural-language question. This is a higher-level command that combines query expansion, semantic search, and AI-generated answers. Requires ANTHROPIC_API_KEY to be set.

python cli.py ask <question> [--top-k N] [--window N] [--name SUBSTR]

Flag	Default	Description
`<question>`	(required)	Natural-language question to answer
`--top-k N`	`5`	Maximum number of search results to retrieve
`--window N`	`0`	Adjacent chunks to include around each search hit
`--name SUBSTR`	—	Restrict to documents whose name contains this substring (case-insensitive)

Pipeline:

Query expansion — an LLM rewrites the question into optimized search keyphrases
Semantic search — the expanded query is searched against the vector store
Answer generation — an LLM synthesizes an answer grounded in the retrieved chunks

Models are configurable via environment variables:

VECTOR_QUERY_MODEL — model for query expansion (default: claude-3-haiku-20240307)
VECTOR_ANSWER_MODEL — model for answering (default: claude-sonnet-4-20250514)

Progress output:

{"type": "progress", "status": "expanding_query", "question": "What models does Docling use?"}
{"type": "progress", "status": "searching", "expanded_query": "Docling ML models layout analysis TableFormer"}
{"type": "progress", "status": "answering", "result_count": 5}

Final result:

{
  "answer": "Docling uses two primary models: a layout analysis model...",
  "expanded_query": "Docling ML models layout analysis TableFormer",
  "sources": [
    {"document": "report", "page": 3, "file_hash": "11465328351749295394"},
    {"document": "report", "page": 5, "file_hash": "11465328351749295394"}
  ]
}

Examples:

# Ask a question across all documents
python cli.py ask "What models does Docling use?"

# Ask with more search depth
python cli.py ask "How are tables extracted?" --top-k 10 --window 1

# Ask within a specific document
python cli.py ask "What are the front setback requirements?" --name "coweta"

`list`

List ingested documents from either the manifest (fast, default) or directly from ChromaDB.

python cli.py list [--name SUBSTR] [--source manifest|vector]

Flag	Default	Description
`--name SUBSTR`	—	Filter by document name or filename substring (case-insensitive)
`--source`	`manifest`	Where to read from: `manifest` or `vector`

`--source` value	Reads from	Fields returned	Best for
`manifest`	`manifest.json` on disk	`page_count`, `ingested_at`, `status`, `chunk_count`, `tags`	Fast everyday listing
`vector`	ChromaDB directly	`chunk_count` per document (live scan)	Verifying what is actually searchable

These two sources can diverge if an ingest fails partway through. Use repair to detect and fix discrepancies.

Examples:

# List all documents (from manifest)
python cli.py list

# Filter by name
python cli.py list --name report

# List from ChromaDB to verify searchable state
python cli.py list --source vector

# List documents in a specific collection
python cli.py --collection legal list

Example output (manifest):

{"count": 1, "source": "manifest", "documents": [
  {"file_hash": "11465328351749295394", "document_name": "report",
   "filename": "report.pdf", "file_extension": ".pdf",
   "page_count": 9, "ingested_at": "2026-03-15T18:36:50",
   "status": "complete", "has_figures": false, "figure_count": 0,
   "tags": {}, "chunk_count": 312}
]}

Example output (vector):

{"count": 1, "source": "vector", "documents": [
  {"file_hash": "11465328351749295394", "document_name": "report",
   "file_extension": ".pdf", "chunk_count": 50}
]}

`status`

Single-call collection health check — shows document count, chunk count, and whether anything needs repair.

python cli.py status

Example output:

{
  "collection": "vector",
  "document_count": 3,
  "chunk_count": 4821,
  "incomplete_count": 0,
  "missing_in_chroma_count": 0,
  "healthy": true
}

Field	Description
`document_count`	Number of documents with `status: complete`
`chunk_count`	Total chunks across all complete documents (read from manifest cache — fast)
`incomplete_count`	Documents with `status: pending` or `failed`
`missing_in_chroma_count`	Documents marked complete but missing from ChromaDB
`healthy`	`true` when `incomplete_count` and `missing_in_chroma_count` are both zero

When healthy is false, run repair --fix to resolve issues.

`collections`

List all collections found in the data directory. A subdirectory is recognized as a collection if it contains a manifest.json file or a documents/ subdirectory.

python cli.py collections

This command uses --data-dir but ignores --collection.

Example output:

{"count": 3, "collections": [
  {"name": "vector", "document_count": 2, "path": "vector_data/vector"},
  {"name": "legal", "document_count": 5, "path": "vector_data/legal"},
  {"name": "research", "document_count": 1, "path": "vector_data/research"}
]}

Field	Description
`name`	Collection name (subdirectory name)
`document_count`	Number of documents with `status: complete`
`path`	Relative path to the collection directory

`repair`

Scan for documents that failed to ingest completely, and optionally re-ingest them automatically from their stored PDF copies.

python cli.py repair [--fix]

Flag	Description
`--fix`	Re-ingest each broken document from its stored PDF copy. Without this flag, the command only reports issues

A document is considered broken if:

Its manifest status is pending or failed, or
It is complete in the manifest but has no chunks in ChromaDB

Dry run (report only):

$ python cli.py repair

{"issue_count": 1,
 "incomplete": [{"file_hash": "...", "status": "failed", "...": "..."}],
 "missing_in_chroma": []}

Fix mode (re-ingest broken documents):

$ python cli.py repair --fix

{"fixed": [{"file_hash": "...", "document_name": "report"}],
 "failed": [],
 "incomplete": [...],
 "missing_in_chroma": [...]}

`tags`

List all tag keys and their distinct values across the collection. Useful for discovering what metadata is in use before filtering queries.

python cli.py tags

Example output:

{
  "tags": {
    "jurisdiction": ["coweta-county-ga", "fayette-county-ga"],
    "state": ["ga"],
    "type": ["report", "zoning-ordinance"]
  }
}

`tag`

Set, update, or remove tags on a document (manifest + vector store). Tags are merged — existing tags not mentioned are preserved.

Auto-normalization: tag values are automatically lowercased, stripped of leading/trailing whitespace, and spaces are replaced with hyphens. A warning is printed to stderr if a value is changed. This ensures consistent values across the collection.

OR syntax: use comma-separated values to express OR matching in queries and filters (e.g. --tag state=GA,FL).

python cli.py tag (--file-hash HASH | --name SUBSTR) [KEY=VALUE ...] [--remove KEY ...]

Flag	Description
`--file-hash HASH`	Hash of the document to tag
`--name SUBSTR`	Substring to match the document name (must be unambiguous)
`KEY=VALUE`	Tag to set in key=value format. Repeatable. Auto-normalized.
`--remove KEY`	Tag key to remove. Repeatable.

Examples:

# Set tags (uppercase values are auto-normalized with a warning)
python cli.py tag --name "coweta" jurisdiction=coweta-county-ga state=GA year=2024
# Warning: tag value 'GA' normalized to 'ga'

# Remove a single tag
python cli.py tag --file-hash 11465328351749295394 --remove state

# Remove multiple tags
python cli.py tag --file-hash 11465328351749295394 --remove jurisdiction --remove year

# Set and remove in the same call
python cli.py tag --file-hash 11465328351749295394 type=report --remove state

# Normalization example: spaces-to-hyphens
python cli.py tag --file-hash 11465328351749295394 "State=Georgia Test"
# Warning: tag value 'Georgia Test' normalized to 'georgia-test'
# → stored as state=georgia-test

Output:

{"status": "tagged", "file_hash": "11465328351749295394", "tags_set": {"state": "ga"}, "tags_removed": []}

`query` and `list` — OR tag filtering

Use comma-separated values in --tag to match documents with any of the given values for a key (OR within a key). Multiple --tag flags still use AND logic across different keys.

# Match documents where type is "research-paper" OR "report"
python cli.py query "setbacks" --tag type=research-paper,report

# Combine OR within a key + AND across keys
python cli.py query "fire code" --tag type=building-code,report --tag state=GA

# List with OR tag filter
python cli.py list --tag state=GA,FL

`figures`

List or export extracted figures for a document. Only available for documents ingested with --extract-figures.

python cli.py figures (--file-hash HASH | --name SUBSTR) [--page N] [--export DIR]

Flag	Description
`--file-hash HASH`	Hash of the document
`--name SUBSTR`	Substring to match the document name (must be unambiguous)
`--page N`	Filter figures to a specific page number
`--export DIR`	Export figure PNGs to a directory

Examples:

# List all figures for a document
python cli.py figures --file-hash 11465328351749295394

# List figures on page 3 only
python cli.py figures --name "report" --page 3

# Export all figure PNGs
python cli.py figures --file-hash 11465328351749295394 --export ./exported_figures

Output (list mode):

{
  "file_hash": "11465328351749295394",
  "figure_count": 8,
  "figures": [
    {
      "id": "11465328351749295394_pic_001",
      "kind": "picture",
      "page_number": 3,
      "caption": "Figure 1: Sketch of Docling's processing pipeline.",
      "image_path": "11465328351749295394_pic_001.png",
      "nearby_chunk_indices": [8, 9]
    }
  ]
}

`rename`

Rename a document in the manifest and vector store. No re-embedding required.

python cli.py rename (--file-hash HASH | --name SUBSTR) NEW_NAME

Flag	Description
`--file-hash HASH`	Hash of the document to rename
`--name SUBSTR`	Substring to match (must be unambiguous)
`NEW_NAME`	New display name for the document

Examples:

python cli.py rename --name "56ce5204" "docling-paper"
python cli.py rename --file-hash 3490373665063567593 "Coweta County Zoning"

Output:

{"status": "renamed", "file_hash": "11465328351749295394", "new_name": "docling-paper"}

`delete`

Remove a document from both the document store (manifest + PDF + figures) and the vector store (all chunks).

python cli.py delete <file_hash>

Flag	Description
`<file_hash>`	(required) The content hash of the document to remove

Examples:

python cli.py delete 11465328351749295394

# Delete from a specific collection
python cli.py --collection legal delete 99887766

Output:

{"status": "deleted", "file_hash": "11465328351749295394"}

Error handling

All errors go to stderr as a single JSON line with exit code 1:

{"error": "Conversion failed: ...", "error_code": "conversion_failed"}

`error_code`	Cause
`conversion_failed`	PDF could not be parsed by docling
`store_failed`	Could not write PDF or manifest to disk
`chunking_failed`	Chunking step raised an exception
`embedding_failed`	ChromaDB write failed
`query_failed`	Vector search failed
`not_found`	`--name` filter matched no documents
`ambiguous_match`	`--name` matched multiple documents (use `--file-hash`)
`list_failed`	Could not read the store
`delete_failed`	Could not delete the document
`rename_failed`	Rename operation failed
`status_failed`	Could not read the store for status check
`query_expansion_failed`	LLM query expansion failed (`ask` only)
`answer_failed`	LLM answer generation failed (`ask` only)

Multi-collection workflow

# Ingest into separate collections
python cli.py --collection research ingest data/paper.pdf
python cli.py --collection legal    ingest data/contract.pdf

# Query within a specific collection only
python cli.py --collection research query "neural architecture"
python cli.py --collection legal    query "termination clause"

# Use a shared network or backup root
python cli.py --data-dir /mnt/shared --collection research ingest data/paper.pdf

# Or set the root once via environment variable
$env:VECTOR_DATA_DIR = "/mnt/shared"
python cli.py --collection research query "neural architecture"

Inspiration

https://docling-project.github.io/docling/examples/visual_grounding/

Name		Name	Last commit message	Last commit date
Latest commit History 11 Commits
.env.example		.env.example
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
Readme.md		Readme.md
agents.py		agents.py
chunker.py		chunker.py
cli.py		cli.py
converter.py		converter.py
document_store.py		document_store.py
example.py		example.py
gui.py		gui.py
models.py		models.py
ptc_zoning.pdf		ptc_zoning.pdf
r2.py		r2.py
rendering.py		rendering.py
requirements.txt		requirements.txt
vector_store.py		vector_store.py

Folders and files

Latest commit

History

Repository files navigation

Vector

Features

Data layout

Modules

models.py

converter.py

rendering.py

chunker.py

document_store.py

vector_store.py

Python API Usage

CLI

Command summary

Global flags

How collections work

ingest

query

ask

list

status

collections

repair

tags

tag

query and list — OR tag filtering

figures

rename

delete

Error handling

Multi-collection workflow

Inspiration

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

`models.py`

`converter.py`

`rendering.py`

`chunker.py`

`document_store.py`

`vector_store.py`

`ingest`

`query`

`ask`

`list`

`status`

`collections`

`repair`

`tags`

`tag`

`query` and `list` — OR tag filtering

`figures`

`rename`

`delete`

Packages