docs/backlog.md has wanted a PDF document-retrieval demo since iteration +3, and #17 has a working one sitting in it that never landed. Worth finishing rather than rewriting.
The example loads a PDF over HTTP (the patent EP0666666B1), splits it per page, indexes the chunks in CrateDBVectorStore, and answers a handful of questions against it. That is the shape most people arrive wanting when they ask what the vector store is actually for, and nothing under examples/ shows it today — vector_search.py fetches a plain text file.
Start from the version under #17 itself, commit c3cba55, where get_documents() is clean. There is a later branch, examples-2, with a further commit on top; that one left the function mid-experiment, with three stacked unreachable returns and a read of the PDF from the working directory instead of from the URL. It looks newer and is worse.
Things to fix on the way in. The module docstring still claims the program uses state_of_the_union.txt and titles itself after Hugging Face, both copied from the sibling example. The pyproject.toml changes on the branch pin against langchain-openai <0.3, langchain-tests ==0.3.7 and pytest <9, twenty months behind, and should simply be dropped. The PEP 723 header says requires-python = ">=3.9" where the project is now 3.10.
One thing worth deciding rather than defaulting: PyPDFLoader comes from langchain_community, which now emits a sunset warning on import pointing at standalone integration packages, and there is no langchain-pypdf on PyPI to move to yet. Shipping on community and revisiting is probably right, but it should be a choice.
Keeping OpenAI embeddings here costs CI only a PDF fetch, which is why this stays separate from the key-free embeddings work.
docs/backlog.mdhas wanted a PDF document-retrieval demo since iteration +3, and #17 has a working one sitting in it that never landed. Worth finishing rather than rewriting.The example loads a PDF over HTTP (the patent EP0666666B1), splits it per page, indexes the chunks in
CrateDBVectorStore, and answers a handful of questions against it. That is the shape most people arrive wanting when they ask what the vector store is actually for, and nothing underexamples/shows it today —vector_search.pyfetches a plain text file.Start from the version under #17 itself, commit c3cba55, where
get_documents()is clean. There is a later branch,examples-2, with a further commit on top; that one left the function mid-experiment, with three stacked unreachable returns and a read of the PDF from the working directory instead of from the URL. It looks newer and is worse.Things to fix on the way in. The module docstring still claims the program uses
state_of_the_union.txtand titles itself after Hugging Face, both copied from the sibling example. Thepyproject.tomlchanges on the branch pin againstlangchain-openai <0.3,langchain-tests ==0.3.7andpytest <9, twenty months behind, and should simply be dropped. The PEP 723 header saysrequires-python = ">=3.9"where the project is now 3.10.One thing worth deciding rather than defaulting:
PyPDFLoadercomes fromlangchain_community, which now emits a sunset warning on import pointing at standalone integration packages, and there is nolangchain-pypdfon PyPI to move to yet. Shipping on community and revisiting is probably right, but it should be a choice.Keeping OpenAI embeddings here costs CI only a PDF fetch, which is why this stays separate from the key-free embeddings work.