Skip to content

[BugFix] Fix MV rewrite serving stale results after Iceberg rollback_to_snapshot (backport #75924)#76170

Merged
wanpengfei-git merged 2 commits into
branch-4.0from
mergify/bp/branch-4.0/pr-75924
Jul 11, 2026
Merged

[BugFix] Fix MV rewrite serving stale results after Iceberg rollback_to_snapshot (backport #75924)#76170
wanpengfei-git merged 2 commits into
branch-4.0from
mergify/bp/branch-4.0/pr-75924

Conversation

@mergify

@mergify mergify Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

By default, StarRocks only rewrites a query against an async materialized view (MV) if every base table is unchanged since the MV's last refresh. mv_rewrite_staleness_second relaxes that: it lets the optimizer still use the MV as long as the elapsed time between the base tables' most recent refresh/modification and the MV's own last refresh is within the configured budget.

isStalenessSatisfied() computed mvStaleness = (baseTableRefreshTimestamp - mvRefreshTimestamp) / 1000 and only rejected the MV as fresh when that value exceeded mv_rewrite_staleness_second. It never checked for the base table's refresh timestamp going backwards relative to the MV's own last refresh.

For Iceberg base tables, IcebergPartitionTraits.maxPartitionRefreshTs() returns the raw currentSnapshot().timestampMillis(). This is normally monotonically increasing, but Iceberg's rollback_to_snapshot/rollback_to_timestamp procedures can move a table's current snapshot - and therefore its timestamp - backwards in time. When that happens, mvStaleness goes negative, which always satisfies mvStaleness > staleness_budget as false, so the MV is treated as fresh and used for rewrite even though the underlying data has been rolled back to a different state than what the MV last captured.

Concretely, getUpdatedPartitionNamesOfExternalTable() / getUpdatedPartitionNamesOfOlapTable() short-circuit to "no partitions need checking" whenever isStalenessSatisfied() returns true, so the bug skips the actual per-partition staleness comparison entirely.

What I'm doing:

Reject the MV outright whenever mvStaleness < 0 instead of falling through to the budget comparison, so a rolled-back base table forces the planner to fall through to the normal per-partition comparison instead of blindly trusting a stale MV.

This only affects MVs that explicitly set mv_rewrite_staleness_second (default is 0, i.e. disabled) and, in practice, is only reachable through Iceberg's rollback_to_snapshot/rollback_to_timestamp procedures, since other connectors' maxPartitionRefreshTs() implementations don't have a documented operation that moves their timestamp backwards.

Fixes #71977

What type of PR is this:

  • BugFix

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5
    • 3.3

Test plan

  • Added MaterializedViewTest#testIsStalenessSatisfiedWithNegativeStaleness and #testIsStalenessSatisfiedWithPositiveStalenessWithinBudget, exercising the new negative-staleness branch and confirming the existing positive-staleness-within-budget path is unaffected.
  • mvn -pl fe-grammar,fe-parser,fe-core -am test-compile clean.
  • mvn -pl fe-core checkstyle:check clean.
  • Both new tests pass: Tests run: 2, Failures: 0, Errors: 0.
    This is an automatic backport of pull request [BugFix] Fix MV rewrite serving stale results after Iceberg rollback_to_snapshot #75924 done by Mergify.

…to_snapshot (#75924)

(cherry picked from commit 86051d9)

# Conflicts:
#	fe/fe-core/src/test/java/com/starrocks/catalog/MaterializedViewTest.java
@mergify

mergify Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Cherry-pick of 86051d9 has failed:

On branch mergify/bp/branch-4.0/pr-75924
Your branch is up to date with 'origin/branch-4.0'.

You are currently cherry-picking commit 86051d97cf.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   fe/fe-core/src/main/java/com/starrocks/catalog/MaterializedView.java

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   fe/fe-core/src/test/java/com/starrocks/catalog/MaterializedViewTest.java

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@mergify mergify Bot added the conflicts label Jul 10, 2026
@wanpengfei-git wanpengfei-git enabled auto-merge (squash) July 10, 2026 04:13
@mergify mergify Bot closed this Jul 10, 2026
auto-merge was automatically disabled July 10, 2026 04:14

Pull request was closed

@mergify

mergify Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@mergify[bot]: Backport conflict, please reslove the conflict and resubmit the pr

@dirtysalt dirtysalt reopened this Jul 11, 2026
Signed-off-by: tqqq <dirtysalt1987@gmail.com>
@wanpengfei-git wanpengfei-git enabled auto-merge (squash) July 11, 2026 02:06

@dirtysalt dirtysalt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@dirtysalt dirtysalt self-assigned this Jul 11, 2026
@wanpengfei-git wanpengfei-git merged commit 3bd7456 into branch-4.0 Jul 11, 2026
30 of 31 checks passed
@wanpengfei-git wanpengfei-git deleted the mergify/bp/branch-4.0/pr-75924 branch July 11, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants