Skip to content

Commit 12eed5b

Browse files
franciscojavierarceoclaude
franciscojavierarceo
and
claude
committed
feat(vector_io): add tenant isolation for vector store metadata via AuthorizedSqlStore
Migrate OpenAIVectorStoreMixin metadata storage from KVStore to AuthorizedSqlStore for row-level tenant isolation. All 9 vector_io providers (2 inline, 7 remote) now accept a policy parameter and optional metadata_store config for SQL-backed, access-controlled metadata. Dual storage support preserves backward compatibility: providers without metadata_store configured continue using KVStore. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
1 parent d30a603 commit 12eed5b

40 files changed

Lines changed: 470 additions & 121 deletions

‎docs/docs/providers/vector_io/inline_faiss.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,9 @@ more details about Faiss in general.
9898
| `persistence` | `KVStoreReference` | No | | |
9999
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
100100
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
101+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
102+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
103+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
101104

102105
## Sample Configuration
103106

‎docs/docs/providers/vector_io/inline_sqlite-vec.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,9 @@ See [sqlite-vec's GitHub repo](https://github.com/asg017/sqlite-vec/tree/main) f
411411
| `persistence` | `KVStoreReference` | No | | Config for KV store backend (SQLite only for now) |
412412
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
413413
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
414+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
415+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
416+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
414417

415418
## Sample Configuration
416419

‎docs/docs/providers/vector_io/remote_chromadb.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,9 @@ See [Chroma's documentation](https://docs.trychroma.com/docs/overview/introducti
8181
| `persistence` | `KVStoreReference` | No | | Config for KV store backend |
8282
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
8383
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
84+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
85+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
86+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
8487

8588
## Sample Configuration
8689

‎docs/docs/providers/vector_io/remote_elasticsearch.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,9 @@ See [Elasticsearch's documentation](https://www.elastic.co/docs/solutions/search
9494
| `persistence` | `KVStoreReference \| None` | No | | Config for KV store backend (SQLite only for now) |
9595
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
9696
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
97+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
98+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
99+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
97100

98101
## Sample Configuration
99102

‎docs/docs/providers/vector_io/remote_infinispan.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -238,6 +238,9 @@ Set the `auth_mechanism` parameter to either `"digest"` or `"basic"`.
238238
| `persistence` | `KVStoreReference` | No | | Config for KV store backend |
239239
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
240240
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
241+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
242+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
243+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
241244

242245
## Sample Configuration
243246

‎docs/docs/providers/vector_io/remote_milvus.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -411,6 +411,9 @@ For more details on TLS configuration, refer to the [TLS setup guide](https://mi
411411
| `persistence` | `KVStoreReference` | No | | Config for KV store backend |
412412
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
413413
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
414+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
415+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
416+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
414417
| `config` | `dict` | No | &#123;&#125; | This configuration allows additional fields to be passed through to the underlying Milvus client. See the [Milvus](https://milvus.io/docs/install-overview.md) documentation for more details about Milvus in general. |
415418

416419
:::note

‎docs/docs/providers/vector_io/remote_pgvector.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,9 @@ See [PGVector's documentation](https://github.com/pgvector/pgvector) for more de
239239
| `persistence` | `KVStoreReference \| None` | No | | Config for KV store backend (SQLite only for now) |
240240
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
241241
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
242+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
243+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
244+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
242245

243246
## Sample Configuration
244247

‎docs/docs/providers/vector_io/remote_qdrant.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ Please refer to the inline provider documentation.
2929
| `persistence` | `KVStoreReference` | No | | |
3030
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
3131
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
32+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
33+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
34+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
3235

3336
## Sample Configuration
3437

‎docs/docs/providers/vector_io/remote_weaviate.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,9 @@ See [Weaviate's documentation](https://weaviate.io/developers/weaviate) for more
7878
| `persistence` | `KVStoreReference \| None` | No | | Config for KV store backend (SQLite only for now) |
7979
| `persistence.namespace` | `str` | No | | Key prefix for KVStore backends |
8080
| `persistence.backend` | `str` | No | | Name of backend from storage.backends |
81+
| `metadata_store` | `SqlStoreReference \| None` | No | | SQL store reference for tenant-isolated vector store metadata |
82+
| `metadata_store.table_name` | `str` | No | | Name of the table to use for the SqlStore |
83+
| `metadata_store.backend` | `str` | No | | Name of backend from storage.backends |
8184

8285
## Sample Configuration
8386

‎src/ogx/core/datatypes.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -973,4 +973,5 @@ def _ensure_backend(reference, expected_set, store_name: str) -> None:
973973
_ensure_backend(stores.responses, sql_backends, "storage.stores.responses")
974974
_ensure_backend(stores.prompts, sql_backends, "storage.stores.prompts")
975975
_ensure_backend(stores.connectors, sql_backends, "storage.stores.connectors")
976+
_ensure_backend(stores.vector_stores, sql_backends, "storage.stores.vector_stores")
976977
return self

0 commit comments

Comments
 (0)