Skip to content

[Enhancement] Align range-colocate scan-range dispatch by ColocateRange index (backport #72278)#72804

Merged
wanpengfei-git merged 1 commit into
branch-4.1from
mergify/bp/branch-4.1/pr-72278
May 6, 2026
Merged

[Enhancement] Align range-colocate scan-range dispatch by ColocateRange index (backport #72278)#72804
wanpengfei-git merged 1 commit into
branch-4.1from
mergify/bp/branch-4.1/pr-72278

Conversation

@mergify

@mergify mergify Bot commented May 6, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

Range-distribution colocate joins emit no-exchange plans (introduced in #71860), but the FE coordinator still dispatched scan ranges by tablet position within a partition (in both OlapScanNode.computeTabletInfo and PlanFragmentBuilder.visitPhysicalOlapScan). Position-based bucketSeq only aligns left/right tablets when both sides preserve identical tablet ordering — and AlterTableClauseAnalyzer.visitSplitTabletClause (no colocate guard) plus auto-reshard can already produce sub-range tablets on range distribution colocate tables, breaking that invariant. Without this fix, range-colocate joins can silently mis-route scan ranges and return wrong results.

This is the runtime-correctness prerequisite for using #71860's plans in production.

What I'm doing:

Source the bucket sequence from the ColocateRange index in ColocateRangeMgr instead of tablet position whenever an OlapScanNode reads a range-distribution lake table that participates in a colocate group. The index is stable across all tables in the colocate group (the i-th range corresponds to the same PACK shard group across every partition/table/database), so left and right scan ranges sharing a colocate range now land on the same fragment instance regardless of tablet insertion order.

  • ColocateRangeMgr.getColocateRangeIndex(grpId, Tuple) — stable bucket-sequence lookup; null value (lower bound -inf) maps to the first range. getColocateRange is reworded to delegate.
  • ColocateRangeUtils.extractColocatePrefix(Range<Tuple>, int) — inverse of expandToFullSortKey; truncates a tablet's full sort-key lower bound to the colocate prefix; returns null if the lower bound is -inf.
  • ColocateRangeUtils.computeRangeColocateBucketSeq(OlapTable, MaterializedIndex, allTabletIdsInOrder, ColocateTableIndex) — the shared utility called from BOTH OlapScanNode.computeTabletInfo and PlanFragmentBuilder.visitPhysicalOlapScan so the legacy and optimizer-built scan paths cannot drift. Returns null for non-range-colocate scans (caller falls back to position-based bucketSeq).
  • ColocateRangeUtils package-private rangeContains helper — bound-aware containment that handles -∞/+∞ and inclusive/exclusive edges (avoids Range.compareTo's overlap semantics).
  • OlapScanNode.getBucketNums() overridden to return the colocate range count for range colocate tables, so ColocatedBackendSelector's bucketSeq < bucketNum invariant holds (RangeDistributionInfo.getBucketNum() is hard-coded to 1).

Fail-closed alignment guard inside the utility: throws IllegalStateException when the colocate group is in an unaligned state — either a tablet whose range is not contained in any single ColocateRange (the spanning-tablet case), or a ColocateRange with no covering tablet in the partition's index (the missing-range case). This is critical because a Range.all() tablet co-existing with a multi-range ColocateRangeMgr would otherwise silently misroute scan ranges.

The check uses containment rather than strict equality so that legitimate intra-colocate splits (which produce sub-range tablets within a single ColocateRange) keep working. When P3's range-aware split + alignment daemon ship, both alignment states (inter-colocate and intra-colocate) are handled by the existing utility without further change.

Test coverage

  • Helper tests: 4 new cases on ColocateRangeMgrTest (index lookup), 6 new cases on ColocateRangeUtilsTest for extractColocatePrefix, 5 new cases for rangeContains.
  • RangeColocateScanRangeDispatchTest (new, 10 cases) using UtFrameUtils.createMinStarRocksCluster(SHARED_DATA):
    • getBucketNums() against real range colocate tables (single-range default, after multi-range injection, non-colocate fallback).
    • computeRangeColocateBucketSeq against real tables (single range, non-colocate returns null, spanning-tablet throws "unaligned").
    • computeRangeColocateBucketSeq against synthetic fixtures (intra-colocate split sub-range tablets share a bucket; missing-range throws "no covering tablet"; missing-TabletRange throws).
    • End-to-end PlanFragmentBuilder test that drives a real range-colocate join via UtFrameUtils.getPlanAndFragment, walks ExecPlan.getScanNodes(), and asserts bucketSeq2locations aligns by ColocateRange.

Follow-up dependency note

P5 (edit-log persistence for ColocateRangeMgr) must precede multi-FE production enablement of P3 (range-aware splits). Without P5, follower FEs in a multi-FE cluster will plan with stale ColocateRangeMgr after splits and produce wrong join results — the new alignment guard catches the leader's view, not divergence between leader and follower.

Fixes #64986

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

…ge index (#72278)

Signed-off-by: xiangguangyxg <xiangguangyxg@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
(cherry picked from commit 075e7b7)
@wanpengfei-git
wanpengfei-git merged commit febc1c3 into branch-4.1 May 6, 2026
35 of 36 checks passed
@wanpengfei-git
wanpengfei-git deleted the mergify/bp/branch-4.1/pr-72278 branch May 6, 2026 05:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants