Skip to content

Use LinkML-Store for embedding generation#22

Merged
caufieldjh merged 1 commit into
mainfrom
linkml-store-embedding-backend
Jun 3, 2026
Merged

Use LinkML-Store for embedding generation#22
caufieldjh merged 1 commit into
mainfrom
linkml-store-embedding-backend

Conversation

@caufieldjh

Copy link
Copy Markdown
Contributor

Summary

  • replace the active CurateGPT embedding backend with LinkML-Store's LLM indexer
  • export LinkML-Store indexed vectors as a stable embeddings CSV column and load them from CSV/DuckDB
  • remove the CurateGPT dependency and add LinkML-Store/llm/tiktoken dependencies
  • document custom embedding models and OpenAI-compatible endpoint plugin setup

Validation

  • poetry run pytest -q -> 59 passed, 3 skipped, 1 upstream LinkML-Store/Pydantic deprecation warning
  • RUN_OPENAI_INTEGRATION=1 poetry run pytest tests/integration/test_generate_embeddings_integration.py -q -> 1 passed, 1 upstream LinkML-Store/Pydantic deprecation warning
  • poetry check -> passed with Poetry metadata deprecation warnings
  • poetry run flake8 --ignore E501,W503 src/trowel/utils/embedding_generation_utils.py src/trowel/utils/embedding_utils.py src/trowel/cli.py tests/test_embedding_generation.py tests/test_embedding_utils.py tests/test_cli_embeddings.py

Closes #20
Refs #14

@caufieldjh caufieldjh merged commit 794ea21 into main Jun 3, 2026
2 checks passed
@caufieldjh caufieldjh deleted the linkml-store-embedding-backend branch June 3, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Switch to use linkml-store instead of curategpt

1 participant