Skip to content

Commit 023348a

Browse files
feat(vector-io): Add the Neo4j vector provider (#6274)
## Summary - Add the `remote::neo4j` VectorIO provider with vector, keyword, hybrid, and graph-aware retrieval. - Store chunks with correct upsert semantics, including stale-metadata replacement and a per-store `chunk_id` uniqueness constraint. - Add deterministic unit and live Neo4j integration coverage, including a standard RAG versus GraphRAG comparison. - Add generated provider documentation, dependencies, and GitHub Actions coverage with a Neo4j 2025.10 service. ## Validation - `uv run pytest tests/unit/providers/vector_io/test_neo4j.py -q` -> 8 passed - `NEO4J_URI=bolt://localhost:7687 NEO4J_USER=neo4j NEO4J_PASSWORD=ogxpassword uv run pytest tests/integration/vector_io/test_neo4j_graph_retrieval.py -q` -> 1 passed - `uv run ruff check ...` -> passed - Workflow YAML parsing -> passed The live integration test uses fixed embeddings and deterministic graph relationships. Standard retrieval returns only the directly matching chunk; GraphRAG returns the related chunk reached through the graph. --------- Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
1 parent 7005c5a commit 023348a

12 files changed

Lines changed: 1528 additions & 1 deletion

File tree

.github/workflows/integration-vector-io-tests.yml

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ jobs:
3838
strategy:
3939
matrix:
4040
# Use a smaller set of providers on PRs to reduce runner contention
41-
vector-io-provider: ${{ github.event_name == 'pull_request' && fromJSON('["inline::faiss", "inline::sqlite-vec", "remote::pgvector"]') || fromJSON('["inline::faiss", "inline::sqlite-vec", "inline::milvus", "remote::chromadb", "remote::pgvector", "remote::weaviate", "remote::qdrant", "remote::elasticsearch", "remote::infinispan"]') }}
41+
vector-io-provider: ${{ github.event_name == 'pull_request' && fromJSON('["inline::faiss", "inline::sqlite-vec", "remote::pgvector", "remote::neo4j"]') || fromJSON('["inline::faiss", "inline::sqlite-vec", "inline::milvus", "remote::chromadb", "remote::pgvector", "remote::weaviate", "remote::qdrant", "remote::elasticsearch", "remote::infinispan", "remote::neo4j"]') }}
4242
python-version: ${{ github.event.schedule == '0 0 * * 1' && fromJSON('["3.12", "3.13"]') || fromJSON('["3.12"]') }}
4343
fail-fast: false # we want to run all tests regardless of failure
4444

@@ -81,6 +81,30 @@ jobs:
8181
-p 5432:5432 \
8282
pgvector/pgvector:pg17
8383
84+
- name: Start Neo4j
85+
if: matrix.vector-io-provider == 'remote::neo4j'
86+
run: |
87+
docker run --rm -d \
88+
--name neo4j \
89+
-p 7474:7474 -p 7687:7687 \
90+
-e NEO4J_AUTH=neo4j/ogxpassword \
91+
neo4j:2025.10
92+
93+
- name: Wait for Neo4j to be ready
94+
if: matrix.vector-io-provider == 'remote::neo4j'
95+
run: |
96+
echo "Waiting for Neo4j to be ready..."
97+
for _ in {1..60}; do
98+
if docker exec neo4j cypher-shell -u neo4j -p ogxpassword 'RETURN 1 AS ok' >/dev/null 2>&1; then
99+
echo "Neo4j is ready!"
100+
exit 0
101+
fi
102+
sleep 2
103+
done
104+
echo "Neo4j failed to start"
105+
docker logs neo4j
106+
exit 1
107+
84108
- name: Wait for PGVector to be ready
85109
if: matrix.vector-io-provider == 'remote::pgvector'
86110
run: |
@@ -239,6 +263,10 @@ jobs:
239263
INFINISPAN_URL: ${{ matrix.vector-io-provider == 'remote::infinispan' && 'http://localhost:11222' || '' }}
240264
INFINISPAN_USERNAME: ${{ matrix.vector-io-provider == 'remote::infinispan' && 'admin' || '' }}
241265
INFINISPAN_PASSWORD: ${{ matrix.vector-io-provider == 'remote::infinispan' && 'password' || '' }}
266+
NEO4J_URI: ${{ matrix.vector-io-provider == 'remote::neo4j' && 'bolt://localhost:7687' || '' }}
267+
NEO4J_USER: ${{ matrix.vector-io-provider == 'remote::neo4j' && 'neo4j' || '' }}
268+
NEO4J_PASSWORD: ${{ matrix.vector-io-provider == 'remote::neo4j' && 'ogxpassword' || '' }}
269+
NEO4J_DATABASE: ${{ matrix.vector-io-provider == 'remote::neo4j' && 'neo4j' || '' }}
242270
run: |
243271
uv run --no-sync \
244272
pytest -sv --stack-config="files=inline::localfs,inference=inline::sentence-transformers?trust_remote_code=true,vector_io=${{ matrix.vector-io-provider }},file_processors=inline::pypdf" \
@@ -275,6 +303,11 @@ jobs:
275303
run: |
276304
docker logs infinispan > infinispan.log
277305
306+
- name: Write Neo4j logs to file
307+
if: ${{ always() && matrix.vector-io-provider == 'remote::neo4j' }}
308+
run: |
309+
docker logs neo4j > neo4j.log
310+
278311
- name: Upload all logs to artifacts
279312
if: ${{ always() }}
280313
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1

docs/docs/building_applications/rag.mdx

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,99 @@ OGX provides OpenAI-compatible RAG capabilities through:
182182
- **Files API**: Document upload and processing using OpenAI's file format
183183
- **Responses API**: Enhanced chat completions with agentic tool calling via file search
184184

185+
## GraphRAG with Neo4j
186+
187+
Standard RAG retrieves chunks that are semantically or lexically similar to a
188+
query. GraphRAG starts with those same vector or keyword results, then follows
189+
relationships in a graph to find related chunks that may not contain the query
190+
terms themselves. This is useful for questions that depend on shared entities,
191+
references, or multi-hop relationships across documents.
192+
193+
OGX provides GraphRAG through the `remote::neo4j` VectorIO provider. The
194+
provider performs the initial vector, keyword, or hybrid search in Neo4j, then
195+
expands the result through connected chunk nodes. In the deterministic example
196+
below, standard retrieval returns a Python chunk, while GraphRAG can also return
197+
a machine-learning chunk connected through the `python` entity.
198+
199+
### Configure Neo4j
200+
201+
Start Neo4j locally:
202+
203+
```bash
204+
docker run -d --name ogx-neo4j \
205+
-p 7474:7474 -p 7687:7687 \
206+
-e NEO4J_AUTH=neo4j/ogxpassword \
207+
neo4j:2025.10
208+
```
209+
210+
Add a Neo4j provider to your stack configuration and enable graph retrieval:
211+
212+
```yaml
213+
providers:
214+
vector_io:
215+
- provider_id: neo4j
216+
provider_type: remote::neo4j
217+
config:
218+
uri: ${env.NEO4J_URI:=bolt://localhost:7687}
219+
user: ${env.NEO4J_USER:=neo4j}
220+
password: ${env.NEO4J_PASSWORD:=}
221+
database: ${env.NEO4J_DATABASE:=neo4j}
222+
graph_retrieval_enabled: true
223+
graph_expansion_depth: 2
224+
graph_max_neighbors: 10
225+
graph_expansion_weight: 0.15
226+
persistence:
227+
backend: kv_default
228+
namespace: vector_io::neo4j
229+
```
230+
231+
For the local container above, set `NEO4J_PASSWORD=ogxpassword` before starting
232+
OGX. Use a secret manager or an environment-specific secret in production.
233+
234+
The provider supports vector, keyword, and hybrid search. `graph_expansion_depth`
235+
is limited to three hops, and `graph_relationship_types` can restrict traversal
236+
to selected relationship types. Neighbors receive a score based on the seed
237+
chunk score multiplied by `graph_expansion_weight`.
238+
239+
### Index Documents and Add Relationships
240+
241+
Create and populate the vector store using the normal Files and Vector Stores
242+
APIs. Select Neo4j when creating the store:
243+
244+
```python
245+
from openai import OpenAI
246+
247+
client = OpenAI(base_url="http://localhost:8321/v1", api_key="none")
248+
vector_store = client.vector_stores.create(
249+
name="knowledge-graph",
250+
extra_body={"provider_id": "neo4j"},
251+
)
252+
253+
# Upload files and attach them to vector_store as shown above.
254+
```
255+
256+
OGX stores chunks and their embeddings, but it does not infer entities or
257+
relationships for you. Add graph structure with the Neo4j driver or
258+
`cypher-shell`. Chunk labels are derived from the vector store identifier; for
259+
example, a store with the identifier `vs_demo` uses the `OGXChunk_vs_demo` label:
260+
261+
```cypher
262+
MERGE (python:Entity {name: 'python'})
263+
MATCH (python_chunk:OGXChunk_vs_demo {chunk_id: 'chunk-python'})
264+
MATCH (ml_chunk:OGXChunk_vs_demo {chunk_id: 'chunk-ml'})
265+
MERGE (python_chunk)-[:MENTIONS]->(python)
266+
MERGE (ml_chunk)-[:MENTIONS]->(python)
267+
```
268+
269+
Use the same Vector Stores search or Responses `file_search` workflow after
270+
relationships are present. With `graph_retrieval_enabled: false`, retrieval is
271+
standard RAG. With it enabled, the Neo4j provider expands the initial results
272+
through the graph and returns the highest-scoring seed and related chunks.
273+
274+
GraphRAG quality depends on the graph structure: add relationships that encode
275+
meaningful connections, keep traversal depth and neighbor limits bounded, and
276+
compare results against standard retrieval before enabling it broadly.
277+
185278
## Configuring Default Embedding Models
186279

187280
To enable automatic vector store creation without specifying embedding models, configure a default embedding model in your config.yaml like so:
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
description: |
3+
[Neo4j](https://neo4j.com/) is a remote graph database provider for OGX VectorIO.
4+
It supports vector search with Neo4j vector indexes, keyword search with full-text
5+
indexes, hybrid search with OGX reranking, and optional graph-aware expansion through
6+
relationships between retrieved chunks.
7+
8+
## Features
9+
10+
- Vector search over chunk embeddings
11+
- Keyword search over chunk text
12+
- Hybrid search using OGX reranking
13+
- Optional graph-aware retrieval through related chunks
14+
- OpenAI-compatible vector store lifecycle through OGX
15+
16+
## Usage
17+
18+
Run Neo4j locally:
19+
20+
```bash
21+
docker run -d --name ogx-neo4j -p 7474:7474 -p 7687:7687 \
22+
-e NEO4J_AUTH=neo4j/ogxpassword neo4j:2025.10
23+
```
24+
sidebar_label: Remote - Neo4J
25+
title: remote::neo4j
26+
---
27+
28+
# remote::neo4j
29+
30+
## Description
31+
32+
33+
[Neo4j](https://neo4j.com/) is a remote graph database provider for OGX VectorIO.
34+
It supports vector search with Neo4j vector indexes, keyword search with full-text
35+
indexes, hybrid search with OGX reranking, and optional graph-aware expansion through
36+
relationships between retrieved chunks.
37+
38+
## Features
39+
40+
- Vector search over chunk embeddings
41+
- Keyword search over chunk text
42+
- Hybrid search using OGX reranking
43+
- Optional graph-aware retrieval through related chunks
44+
- OpenAI-compatible vector store lifecycle through OGX
45+
46+
## Usage
47+
48+
Run Neo4j locally:
49+
50+
```bash
51+
docker run -d --name ogx-neo4j -p 7474:7474 -p 7687:7687 \
52+
-e NEO4J_AUTH=neo4j/ogxpassword neo4j:2025.10
53+
```
54+
55+
56+
## Configuration
57+
58+
| Field | Type | Required | Default | Description |
59+
|-------|------|----------|---------|-------------|
60+
| `uri` | `str` | No | bolt://localhost:7687 | Neo4j Bolt URI. |
61+
| `user` | `str` | No | neo4j | Neo4j username. |
62+
| `password` | `SecretStr \| None` | No | | Neo4j password. |
63+
| `database` | `str` | No | neo4j | Neo4j database name. |
64+
| `index_prefix` | `str` | No | ogx | Prefix for Neo4j vector and full-text indexes. |
65+
| `graph_retrieval_enabled` | `bool` | No | False | Whether to expand initial retrieval results through graph relationships. |
66+
| `graph_expansion_depth` | `int` | No | 1 | Maximum relationship traversal depth for graph-aware retrieval. |
67+
| `graph_max_neighbors` | `int` | No | 10 | Maximum graph-expanded neighbor chunks to consider per query. |
68+
| `graph_expansion_weight` | `float` | No | 0.15 | Score multiplier applied to graph-expanded chunks relative to seed chunk scores. |
69+
| `graph_relationship_types` | `list[str] \| None` | No | | Relationship types used for graph expansion. When omitted, all relationships are considered. |
70+
| `persistence` | `KVStoreReference` | No | | Config for KV store backend. |
71+
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
72+
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
73+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata. |
74+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
75+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
76+
77+
## Sample Configuration
78+
79+
```yaml
80+
uri: ${env.NEO4J_URI:=bolt://localhost:7687}
81+
user: ${env.NEO4J_USER:=neo4j}
82+
password: ${env.NEO4J_PASSWORD:=}
83+
database: ${env.NEO4J_DATABASE:=neo4j}
84+
graph_retrieval_enabled: false
85+
graph_expansion_depth: 1
86+
graph_max_neighbors: 10
87+
graph_expansion_weight: 0.15
88+
persistence:
89+
namespace: vector_io::neo4j
90+
backend: kv_default
91+
```

0 commit comments

Comments
 (0)