[Enhancement] Align range-colocate scan-range dispatch by ColocateRange index (backport #72278)#72804
Merged
Merged
Conversation
xiangguangyxg
approved these changes
May 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
Range-distribution colocate joins emit no-exchange plans (introduced in #71860), but the FE coordinator still dispatched scan ranges by tablet position within a partition (in both
OlapScanNode.computeTabletInfoandPlanFragmentBuilder.visitPhysicalOlapScan). Position-based bucketSeq only aligns left/right tablets when both sides preserve identical tablet ordering — andAlterTableClauseAnalyzer.visitSplitTabletClause(no colocate guard) plus auto-reshard can already produce sub-range tablets on range distribution colocate tables, breaking that invariant. Without this fix, range-colocate joins can silently mis-route scan ranges and return wrong results.This is the runtime-correctness prerequisite for using #71860's plans in production.
What I'm doing:
Source the bucket sequence from the
ColocateRangeindex inColocateRangeMgrinstead of tablet position whenever anOlapScanNodereads a range-distribution lake table that participates in a colocate group. The index is stable across all tables in the colocate group (the i-th range corresponds to the same PACK shard group across every partition/table/database), so left and right scan ranges sharing a colocate range now land on the same fragment instance regardless of tablet insertion order.ColocateRangeMgr.getColocateRangeIndex(grpId, Tuple)— stable bucket-sequence lookup;nullvalue (lower bound-inf) maps to the first range.getColocateRangeis reworded to delegate.ColocateRangeUtils.extractColocatePrefix(Range<Tuple>, int)— inverse ofexpandToFullSortKey; truncates a tablet's full sort-key lower bound to the colocate prefix; returnsnullif the lower bound is-inf.ColocateRangeUtils.computeRangeColocateBucketSeq(OlapTable, MaterializedIndex, allTabletIdsInOrder, ColocateTableIndex)— the shared utility called from BOTHOlapScanNode.computeTabletInfoandPlanFragmentBuilder.visitPhysicalOlapScanso the legacy and optimizer-built scan paths cannot drift. Returnsnullfor non-range-colocate scans (caller falls back to position-based bucketSeq).ColocateRangeUtilspackage-privaterangeContainshelper — bound-aware containment that handles-∞/+∞and inclusive/exclusive edges (avoidsRange.compareTo's overlap semantics).OlapScanNode.getBucketNums()overridden to return the colocate range count for range colocate tables, soColocatedBackendSelector'sbucketSeq < bucketNuminvariant holds (RangeDistributionInfo.getBucketNum()is hard-coded to 1).Fail-closed alignment guard inside the utility: throws
IllegalStateExceptionwhen the colocate group is in an unaligned state — either a tablet whose range is not contained in any single ColocateRange (the spanning-tablet case), or a ColocateRange with no covering tablet in the partition's index (the missing-range case). This is critical because aRange.all()tablet co-existing with a multi-rangeColocateRangeMgrwould otherwise silently misroute scan ranges.The check uses containment rather than strict equality so that legitimate intra-colocate splits (which produce sub-range tablets within a single ColocateRange) keep working. When P3's range-aware split + alignment daemon ship, both alignment states (inter-colocate and intra-colocate) are handled by the existing utility without further change.
Test coverage
ColocateRangeMgrTest(index lookup), 6 new cases onColocateRangeUtilsTestforextractColocatePrefix, 5 new cases forrangeContains.RangeColocateScanRangeDispatchTest(new, 10 cases) usingUtFrameUtils.createMinStarRocksCluster(SHARED_DATA):getBucketNums()against real range colocate tables (single-range default, after multi-range injection, non-colocate fallback).computeRangeColocateBucketSeqagainst real tables (single range, non-colocate returnsnull, spanning-tablet throws "unaligned").computeRangeColocateBucketSeqagainst synthetic fixtures (intra-colocate split sub-range tablets share a bucket; missing-range throws "no covering tablet"; missing-TabletRangethrows).PlanFragmentBuildertest that drives a real range-colocate join viaUtFrameUtils.getPlanAndFragment, walksExecPlan.getScanNodes(), and assertsbucketSeq2locationsaligns by ColocateRange.Follow-up dependency note
P5 (edit-log persistence for
ColocateRangeMgr) must precede multi-FE production enablement of P3 (range-aware splits). Without P5, follower FEs in a multi-FE cluster will plan with staleColocateRangeMgrafter splits and produce wrong join results — the new alignment guard catches the leader's view, not divergence between leader and follower.Fixes #64986
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check:
This is an automatic backport of pull request [Enhancement] Align range-colocate scan-range dispatch by ColocateRange index #72278 done by Mergify.