Skip to content

Accommodate ORM schema reorg for search/browse queries - #207

Merged
skearnes merged 4 commits into
open-reaction-database:mainfrom
skearnes:accommodate-orm-schema-reorg
Jul 24, 2026
Merged

Accommodate ORM schema reorg for search/browse queries#207
skearnes merged 4 commits into
open-reaction-database:mainfrom
skearnes:accommodate-orm-schema-reorg

Conversation

@skearnes

@skearnes skearnes commented Jun 29, 2026

Copy link
Copy Markdown
Member

Summary

ord-schema's 0.8.0 release reorganizes the ORM into role-based schemas (open-reaction-database/ord-schema#859) and splits dataset loading into separate ingest and derivation stages (open-reaction-database/ord-schema#860, open-reaction-database/ord-schema#861):

  • public.reactions — the served Reaction proto payload, keyed by reaction_id (moved off ord.reaction.proto).
  • public.datasets — per-dataset metadata num_reactions / submitted_at (moved off ord.dataset).
  • derived.* — generated SMILES + rdkit_*_id links: reaction_smiles, compound_smiles, product_compound_smiles (moved off ord.reaction / ord.compound / ord.product_compound).
  • ord.* — now a pure search index (decomposed message tables only).
  • Loading is now two stagesadd_dataset / add_parquet_dataset ingest ord.* + public.*, and update_derived_data populates the derived.* SMILES / RDKit links / reaction classes.

This PR updates the search/browse queries to read from the new locations and updates the test-fixture loader to the two-stage API.

Changes

  • ReactionSmartsQuery / ReactionComponentQuery (queries.py) — join through derived.reaction_smiles / derived.compound_smiles / derived.product_compound_smiles to reach the rdkit_*_id links, replacing the dropped ord.reaction.rdkit_reaction_id / compound.rdkit_mol_id columns.
  • fetch_reactions — reads proto from public.reactions.
  • _fetch_dataset_most_used_smiles — joins the derived SMILES tables for the smiles column (new smiles_table / smiles_foreign_key params).
  • get_datasets / get_dataset (search.py) — join public.datasets for num_reactions / submitted_at.
  • setup_test_postgres (testing.py) — the removed add_dataset(..., rdkit_cartridge=...) keyword is gone; ingest with add_dataset(dataset, session) then derive with update_derived_data(dataset.dataset_id, session, rdkit_cartridge=...), mirroring the ord-schema ORM conftest and preserving the rdkit-cartridge gating.
  • queries_test.py — the _max_similarities helper mirrors _mols_join, updated to the same derived-table joins.
  • pyproject.toml — bump the pin to ord-schema[orm]>=0.8,<0.9.

Reaction classes (ord-schema#854)

No changes needed: ord-interface does not read the reaction class/name fields anywhere (no query, search, or frontend reference), so #854 — and its move to derived.reaction_classes in #859 — is transparent here.

Correctness note

All joins to the derived.* tables are INNER, which preserves the prior NULL-column filtering: update_derived_tables inserts a derived row only when a SMILES can be generated, so un-derivable entities are dropped exactly as the old rdkit_*_id IS NULL / smiles IS NOT NULL paths did. The API response shapes (DatasetInfo, QueryResult) are unchanged, so no frontend changes are needed.

Release status ✅

ord-schema 0.8.0 is published, so this is unblocked:

  • main merged in and uv.lock regenerated (uv lock): ord-schema 0.7.1 → 0.8.0, plus the new uuid6 transitive dependency.
  • The ty check pre-commit hook — previously skipped because it could not resolve >=0.8 — now passes, along with ruff check, ruff format, and addlicense.
  • The DB-backed pytest suite (Postgres + RDKit cartridge) runs in CI.

🤖 Generated with Claude Code

Greptile Summary

Updates the interface for the ord-schema 0.8 ORM reorganization.

  • Routes reaction, compound, and product queries through the new derived.* tables.
  • Reads served reaction payloads and dataset metadata from public.*.
  • Updates test-database setup to run separate ingestion and derivation stages.
  • Raises the ord-schema dependency range to 0.8 and refreshes the lockfile.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failures remain within the follow-up review scope.

No blocking failures remain.

Important Files Changed

Filename Overview
ord_interface/api/queries.py Migrates reaction payload, structural search, and SMILES-statistics queries to the reorganized public and derived schemas.
ord_interface/api/search.py Joins public dataset metadata into dataset listing and lookup responses.
ord_interface/api/testing.py Updates fixture loading to perform ingestion followed by derived-data generation.
ord_interface/api/queries_test.py Aligns similarity-test joins with the new derived RDKit-link tables.
pyproject.toml Updates the supported ord-schema ORM release range to 0.8.
uv.lock Resolves ord-schema 0.8.0 and its new uuid6 transitive dependency.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  API[Search and browse API] --> IDX[ord.* search index]
  IDX --> DER[derived.* SMILES and RDKit links]
  IDX --> PUB[public.reactions and public.datasets]
  LOAD[Fixture loader] --> INGEST[add_dataset]
  INGEST --> DERIVE[update_derived_data]
  DERIVE --> DER
Loading

Reviews (2): Last reviewed commit: "Regenerate uv.lock for ord-schema 0.8.0" | Re-trigger Greptile

skearnes and others added 4 commits June 28, 2026 20:32
…uted)

ord-schema's upcoming 0.8 release reorganizes the ORM into role-based schemas:
the served Reaction proto moves to public.reactions, per-dataset metadata
(num_reactions, submitted_at) to public.datasets, and the generated SMILES plus
rdkit_*_id links to the derived.* tables (reaction_smiles, compound_smiles,
product_compound_smiles). The ord.* schema is now a pure search index.

Update the search/browse queries to read from the new locations:
- ReactionSmartsQuery / ReactionComponentQuery join ord.reaction (and the
  compound tables) through the derived SMILES tables to reach the rdkit_*_id
  links instead of the dropped ord.reaction.rdkit_reaction_id /
  compound.rdkit_mol_id columns.
- fetch_reactions reads proto from public.reactions.
- The dataset-stats aggregation joins the derived SMILES tables for the SMILES.
- get_datasets / get_dataset join public.datasets for num_reactions /
  submitted_at.

All joins to the derived tables are INNER, mirroring the prior NULL-column
filtering: update_derived_tables inserts a derived row only when a SMILES can be
generated, so un-derivable entities are dropped exactly as before.

Bump the ord-schema pin to >=0.8,<0.9. uv.lock must be regenerated once 0.8.0
is published.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ord-schema 0.8 splits dataset loading into ingest (add_dataset) and derivation
(update_derived_data); update the test-postgres helper to call both, mirroring the
ord-schema ORM conftest and preserving the rdkit-cartridge gating.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
0.8.0 is now published, so resolve the >=0.8,<0.9 pin against it (drops the
stale 0.7.1 lock) and pull in uuid6, the new transitive dependency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@skearnes
skearnes marked this pull request as ready for review July 24, 2026 02:44
@skearnes
skearnes merged commit c12124e into open-reaction-database:main Jul 24, 2026
16 checks passed
@skearnes
skearnes deleted the accommodate-orm-schema-reorg branch July 24, 2026 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant