[BugFix] Fix Delta Lake and Kudu un-partitioned materialized view query rewrite#76359
Merged
Merged
Conversation
|
@codex review |
Contributor
Module-risk briefing for Codex reviewThis is review focus, not a finding list. Validate against the diff. High-confidence risks
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 412388bd87
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
GavinMar
force-pushed
the
fix_mv_rewrite_failed
branch
from
412388b to
5afae0a
Compare
…ry rewrite DeltaLakePartitionTraits/KuduPartitionTraits.getUpdatedPartitionNames returned null because partition-change tracking is not implemented for these connectors. The MV timeliness arbiter treats a null base-table update info as "unknown" and forces a full refresh, which marks the materialized view stale and disables query rewrite in the default CHECKED consistency mode. Return an empty set instead of null (i.e. "no updated partitions detected"), matching the behavior of other connectors that do not yet implement partition-change tracking (e.g. Hudi), so a fresh MV over a Delta Lake or Kudu table stays eligible for query rewrite. Add unit tests asserting the MV timeliness arbiter reports NO_REFRESH (not FULL) for such MVs in the default CHECKED consistency mode. Signed-off-by: GavinMar <yangguansuo@starrocks.com>
GavinMar
force-pushed
the
fix_mv_rewrite_failed
branch
from
5afae0a to
aa1d080
Compare
GavinMar
changed the title
Contributor
[Java-Extensions Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
Contributor
[FE Incremental Coverage Report]✅ pass : 2 / 2 (100.00%) file detail
|
Contributor
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
|
Youngwb
approved these changes
Jul 15, 2026
stephen-shelby
approved these changes
Jul 15, 2026
Contributor
|
@Mergifyio backport branch-4.1 |
Contributor
✅ Backports have been createdDetails
|
23 tasks
Contributor
Author
|
@mergify backport branch-4.0 |
Contributor
✅ Backports have been createdDetails
|
23 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
Materialized views defined on a Delta Lake or Kudu external table fail query rewrite. After the MV is refreshed successfully and is active, a query that matches the MV definition is not rewritten to use the MV; instead it scans the base table directly.
Root cause:
DeltaLakePartitionTraits/KuduPartitionTraitsgetUpdatedPartitionNames()returnsnullbecause partition-change tracking is not implemented for these connectors. The MV timeliness arbiter (MVTimelinessNonPartitionArbiter) interprets anullbase-table update info as "unknown" and returnsMvUpdateInfo.fullRefresh, which marks the MV stale and disables query rewrite in the defaultCHECKEDconsistency mode.This regressed after #73644, which changed the arbiter to treat
nullas "needs full refresh". Before that changenullfell through tonoRefresh, so a fresh external-table MV was eligible for rewrite (see #57686, which originally fixed this for Delta Lake and added thetest_mv_deltalake_rewriteSQL case). Reproduces on branch-4.1.What I'm doing:
DeltaLakePartitionTraitsandKuduPartitionTraitsgetUpdatedPartitionNames()now return an empty set instead ofnull(i.e. "no updated partitions detected"), matching the behavior of other connectors that do not yet implement partition-change tracking (e.g. Hudi). This keeps a fresh Delta Lake / Kudu MV eligible for query rewrite inCHECKEDmode. Two unit tests assert the MV timeliness arbiter reportsNO_REFRESH(notFULL) for such MVs in the defaultCHECKEDconsistency mode.Verified on a branch-4.1.3 cluster: in the default
CHECKEDmode, the plan for the MV-matching query changes from a directDeltaLakeScanNodeon the base table toOlapScanNode TABLE: test_mv1(rewrite hits the MV).Fixes StarRocks/StarRocksTest#11409
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: