Skip to content

add milvus support#187

Merged
ArnavAgrawal03 merged 1 commit into
mainfrom
migrate-to-milvus
Jun 5, 2025
Merged

add milvus support#187
ArnavAgrawal03 merged 1 commit into
mainfrom
migrate-to-milvus

Conversation

@ArnavAgrawal03

Copy link
Copy Markdown
Collaborator

No description provided.

@jazzberry-ai

jazzberry-ai Bot commented Jun 5, 2025

Copy link
Copy Markdown

Bug Report

Name Severity Example test case Description
Missing Milvus URI/API Key validation in core/services_init.py Medium Set VECTOR_STORE_PROVIDER to "milvus", but do not define MILVUS_URI in the environment. Run the application. When VECTOR_STORE_PROVIDER is set to "milvus", the code does not validate the presence of the MILVUS_URI and MILVUS_API_KEY environment variables in core/services_init.py. This will cause a runtime exception when MilvusClient is initialized without the required credentials. A similar check as done for POSTGRES_URI should be added for Milvus.
Invalid Milvus collection schema in core/vector_store/milvus_multivector_store.py High Set MULTIVECTOR_PROVIDER to "milvus". Run the application and attempt to ingest a document. The _create_collection method in core/vector_store/milvus_multivector_store.py creates a Milvus collection schema where both the collection itself (auto_id=True in create_schema) and the "pk" field have auto_id=True. Milvus only allows one auto-id field per collection. This will cause the collection creation to fail, preventing the application from storing multi-vector embeddings in Milvus.
Missing batch size validation when data_rows is empty in MilvusMultiVectorStore Low Configure a large MILVUS_BATCH_SIZE in the settings. Attempt to ingest an empty document, resulting in an empty data_rows list. In MilvusMultiVectorStore.store_embeddings, the safe_batch_size is calculated dynamically using _estimate_batch_size based on a sample row. However, if the data_rows list is empty (e.g., due to an empty document or filtering), the _estimate_batch_size function is never called. The code then uses the configured batch_size which may be too large for the Milvus instance or even be zero. It should add a check to ensure safe_batch_size is at least 1 in this scenario.

Comments? Email us.

@ArnavAgrawal03 ArnavAgrawal03 merged commit 8f20d00 into main Jun 5, 2025
2 checks passed
@ArnavAgrawal03 ArnavAgrawal03 deleted the migrate-to-milvus branch June 5, 2025 02:40
ArnavAgrawal03 added a commit that referenced this pull request Jun 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant