Skip to content

perf: predicate push-down for Expr clauses + mixed rule optimization (#207, #206)#249

Merged
adityamukho merged 12 commits into
mainfrom
feature/issues-207-206-predicate-pushdown
May 15, 2026
Merged

perf: predicate push-down for Expr clauses + mixed rule optimization (#207, #206)#249
adityamukho merged 12 commits into
mainfrom
feature/issues-207-206-predicate-pushdown

Conversation

@adityamukho

Copy link
Copy Markdown
Collaborator

Summary

  • perf(post-1.0): predicate push-down for expression clauses #207: Extends optimizer::plan() to accept Vec<WhereClause> (Pattern + Expr) and return an interleaved plan. Each Expr predicate clause is positioned at the earliest point where all its variables are bound by preceding patterns, rather than being applied as a post-pass over the full binding set.
  • perf(post-1.0): optimize mixed rule evaluation #206: Routes the StratifiedEvaluator mixed-rules path through the updated plan(), giving mixed rules both selectivity-based pattern ordering and Expr push-down with no separate implementation.

Changes

src/query/datalog/optimizer.rs

  • New expr_vars() helper — recursively collects ?foo variable names from an Expr tree
  • New pattern_bound_vars() helper — collects variables bound by a Pattern's EAV slots
  • plan() signature changed: Vec<Pattern> → Vec<(Pattern, IndexHint)> becomes Vec<WhereClause> → Vec<(WhereClause, Option<IndexHint>)>
  • Push-down algorithm: after sorting patterns by selectivity, each Expr is inserted at pos + 1 after the last pattern that completes its variable set

src/query/datalog/matcher.rs

  • New pub(crate) match_with_hint_seeded() — uses the index hint for the first pattern (unit seed), falls back to hash-join for subsequent patterns; used by the executor/evaluator plan loops

src/query/datalog/executor.rs

  • execute_query() and execute_query_with_rules(): replaced two-phase match-then-filter with a single ordered loop processing Pattern (join) and Expr (filter/extend) clauses in plan order
  • Removed both top-level apply_expr_clauses post-passes
  • Not/NotJoin/Or/OrJoin handling is unchanged

src/query/datalog/evaluator.rs

  • StratifiedEvaluator mixed-rules loop now routes through plan() instead of bypassing it, gaining both pattern selectivity ordering and Expr push-down

benches/minigraf_bench.rs

  • New query/predicate_pushdown benchmark group (1K/10K/100K scales, 90th-percentile threshold → ~10% pass rate)

Test plan

  • cargo test — all 598+ tests pass
  • cargo clippy -- -D warnings — no warnings
  • cargo bench --bench minigraf_bench -- query/predicate_pushdown — benchmark runs without panic

Follow-up

🤖 Generated with Claude Code

adityamukho and others added 12 commits May 15, 2026 15:19
…le optimization)

Covers issues #207 and #206: extends optimizer::plan() to accept Expr
clauses and interleave them at the earliest position where their variables
are bound, and routes the StratifiedEvaluator mixed-rules path through
the updated planner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d rules)

TDD plan covering optimizer.rs, matcher.rs, executor.rs, evaluator.rs,
and a new Criterion benchmark. 9 tasks, each with exact file paths,
code blocks, and expected test output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…positioning

Adds expr_vars() and pattern_bound_vars() helpers. plan() now accepts
Vec<WhereClause> (Pattern + Expr) and returns Vec<(WhereClause, Option<IndexHint>)>.
Each Expr is inserted at the earliest position after all its variables are
bound by preceding patterns. No-variable Exprs and Exprs with unbound
variables are appended at the end.

Closes part of #207.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ution

pub(crate) method used by the executor and evaluator plan loops added
in #207: applies an index hint for the first pattern (unit seed) and
falls back to hash-join for subsequent patterns.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Task B refactored plan() to accept Vec<WhereClause> and return
Vec<(WhereClause, Option<IndexHint>)>, but executor.rs wasn't updated.
Extract pattern clauses with their hints to unblock Task C testing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…query_with_rules()

Replaces the two-phase pattern-match-then-expr-filter with a single
ordered loop in both query execution paths. Pattern clauses expand
bindings via match_with_hint_seeded(); Expr clauses filter/extend
inline at their push-down position. Removes both top-level
apply_expr_clauses post-passes. Not/NotJoin/Or/OrJoin handling
is unchanged.

Also suppresses dead-code on match_patterns_with_hints and get_patterns
(retained for future use), fixes collapsible_if and indexing_slicing
warnings introduced by the new code in optimizer.rs and matcher.rs.

Closes part of #207.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Use the Binding type alias instead of spelling out the full HashMap
type in the inline plan loops. Fix the evaluate_branch doc comment
which incorrectly stated it mirrored the top-level execute_query
order (the top-level now uses an interleaved push-down loop).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…down

StratifiedEvaluator mixed-rules loop now collects Pattern + Expr clauses
and calls optimizer::plan(), processing them in the same ordered loop as
the executor. Expr clauses are pushed down to the earliest valid position
in the rule body. Not/NotJoin filtering is unchanged.

Closes #206.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ing it

Replace unwrap_or_default() with ? so storage read failures surface
as errors rather than silently producing an empty candidate set.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Measures multi-pattern query with a selective [(> ?v N)] predicate at
1K/10K/100K fact scales. Threshold set at 90th percentile (10% pass rate).
Provides baseline for evaluating push-down gains.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@codecov

codecov Bot commented May 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.14634% with 26 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/query/datalog/executor.rs 74.60% 16 Missing ⚠️
src/query/datalog/evaluator.rs 86.66% 6 Missing ⚠️
src/query/datalog/optimizer.rs 92.00% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@adityamukho adityamukho merged commit bb43871 into main May 15, 2026
36 checks passed
@adityamukho adityamukho deleted the feature/issues-207-206-predicate-pushdown branch May 15, 2026 17:11
adityamukho added a commit that referenced this pull request May 17, 2026
Wave 2 (PRs #249, #251, #253) was missing from CHANGELOG.md, ROADMAP.md,
and TEST_COVERAGE.md:

- CHANGELOG.md: add Wave 2 section between Wave 3 and Wave 1 entries
- ROADMAP.md: add Wave 2 to timeline between Wave 1 and Wave 3
- TEST_COVERAGE.md: add Wave 2 completion status note (no new tests —
  all optimizer + benchmark work covered by existing 850 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant