Skip to content

[Refactor] Move colocation bucket dispatch onto AbstractOlapTableScanNode#76393

Open
banmoy wants to merge 1 commit into
StarRocks:mainfrom
banmoy:refactor_colocate_olap_scan_node
Open

[Refactor] Move colocation bucket dispatch onto AbstractOlapTableScanNode#76393
banmoy wants to merge 1 commit into
StarRocks:mainfrom
banmoy:refactor_colocate_olap_scan_node

Conversation

@banmoy

@banmoy banmoy commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

The colocation bucket-dispatch contract — the bucket-sequence-to-locations map, the
bucket-count computation, and the tablet-id-to-bucket-sequence fill — lived inside
OlapScanNode, and ColocatedBackendSelector reached into the concrete OlapScanNode
(and directly into a public bucketSeq2locations field) instead of depending on an
abstraction. Hoisting the contract lets any OLAP scan that takes part in colocation
reuse the same machinery, and lets the selector depend on the base type rather than a
concrete node and its public field.

What I'm doing:

Move the colocation bucket-dispatch contract onto AbstractOlapTableScanNode:

  • Add getBucketSeqToLocations() as an optional operation that throws
    UnsupportedOperationException by default; a scan that takes part in colocation
    overrides it to return its own map. Scans that do not (e.g. MetaScanNode) simply
    inherit the default and never call it.
  • Add the stateless static helpers computeBucketNums(...) and
    fillTabletId2BucketSeq(...).
  • OlapScanNode's bucketSeq2locations map is now private final, reached only through
    the getter; getBucketNums() delegates to computeBucketNums(...).
  • ColocatedBackendSelector takes AbstractOlapTableScanNode and reads the map through
    the getter, instead of the concrete OlapScanNode and its public field.

No behavior change: computeBucketNums reproduces the former
getBucketNums/getRangeDistributionBucketNums logic verbatim (the instance reads are
now explicit parameters); fillTabletId2BucketSeq moves unchanged and still resolves for
existing OlapScanNode.fillTabletId2BucketSeq callers via inherited static access;
getBucketSeqToLocations() returns the same map the selector previously read directly.
Existing colocate tests were updated to reach the map through the getter and otherwise
pass unchanged (ColocateJoinTest, RangeColocateScanDispatchTest,
RangeColocateScanRangeDispatchTest, ColocatedBackendSelectorTest,
DefaultSharedDataWorkerProviderTest).

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch

@mergify mergify Bot assigned banmoy Jul 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Module-risk briefing for Codex review

This is review focus, not a finding list. Validate against the diff.

High-confidence risks

  • Module: planner
    Changed files: fe/fe-core/src/main/java/com/starrocks/planner/ColocateOlapScanNode.java, fe/fe-core/src/main/java/com/starrocks/planner/OlapScanNode.java
    Historical failure pattern: recurring "two lists threaded by positional index" bugs around OlapScanNode bucket-count/partition-id computation — a prior fix found pruned sub-partitions silently desyncing partition-id lists from consumers like bucket-count lookup.
    Review focus: this PR moves getBucketNums/getRangeDistributionBucketNums and fillTabletId2BucketSeq out of OlapScanNode into static methods on the new ColocateOlapScanNode base, now taking selectedLogicalPartitionIds, selectedPhysicalPartitions, and tabletId2BucketSeq as explicit parameters instead of reading instance fields directly. Verify the extraction preserves exact parameter correspondence (no swapped/stale list) between getSelectedPartitionIds()/getSelectedPhysicalPartitions() and what's passed into computeBucketNums, especially under pruned/empty sub-partitions.
    Evidence: [BugFix] Fix query cache normalization crash for tables with sub-partitions #75789 (from 20 merged bugfixes)
  • Module: qe
    Changed files: fe/fe-core/src/main/java/com/starrocks/qe/ColocatedBackendSelector.java
    Historical failure pattern: ColocatedBackendSelector.Assignment previously accumulated stale scan-range entries across incremental scheduling batches for buckets absent from the current batch, causing redundant/inconsistent re-scanning.
    Review focus: this PR reroutes every read in ColocatedBackendSelector (computeScanRangeAssignment, createBucketIterator, NormalBucketSequenceIterator, BalancerBucketSequenceIterator) from the raw OlapScanNode.bucketSeq2locations field to scanNode.getBucketSeqToLocations() via the new ColocateOlapScanNode abstraction. Confirm this indirection doesn't change assignment-accumulation semantics across successive batches — exercise with a colocate scan whose bucket set changes between batches.
    Evidence: [BugFix] Fix stale scan ranges accumulation in incremental batch assignment #71789 (from 20 merged bugfixes)

@CelerData-Reviewer

Copy link
Copy Markdown

@codex review

1 similar comment
@CelerData-Reviewer

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: a240b7ab2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…Node

The colocation bucket-dispatch contract -- the bucket-sequence-to-locations map, the
bucket-count computation, and the tablet-id-to-bucket-sequence fill -- lived inside
OlapScanNode, and ColocatedBackendSelector reached into the concrete OlapScanNode (and
directly into a public bucketSeq2locations field) instead of depending on an abstraction.

Move the contract onto AbstractOlapTableScanNode: getBucketSeqToLocations() as an optional
operation that throws by default (a scan that takes part in colocation overrides it), plus
the stateless static helpers computeBucketNums(...) and fillTabletId2BucketSeq(...).
OlapScanNode's bucketSeq2locations map is now private final, reached only through the
getter, and getBucketNums() delegates to computeBucketNums. ColocatedBackendSelector
depends on AbstractOlapTableScanNode and reads the map through the getter, so it no longer
couples to the concrete OlapScanNode. A scan node that does not take part in colocation
(e.g. MetaScanNode) simply does not override the getter.

No behavior change: computeBucketNums reproduces the former
getBucketNums/getRangeDistributionBucketNums logic verbatim (the instance reads are now
explicit parameters); fillTabletId2BucketSeq moves unchanged and still resolves for
existing OlapScanNode.fillTabletId2BucketSeq callers via inherited static access; and
getBucketSeqToLocations returns the same map the selector previously read directly.
Existing colocate tests reach the map through the getter.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
@banmoy banmoy force-pushed the refactor_colocate_olap_scan_node branch from a240b7a to 70cb258 Compare July 15, 2026 06:43
@banmoy banmoy changed the title [Refactor] Extract ColocateOlapScanNode base for colocation bucket dispatch [Refactor] Move colocation bucket dispatch onto AbstractOlapTableScanNode Jul 15, 2026
@CelerData-Reviewer

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 70cb2582d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

Copy link
Copy Markdown
Contributor

[Java-Extensions Incremental Coverage Report]

pass : 0 / 0 (0%)

@github-actions

Copy link
Copy Markdown
Contributor

[FE Incremental Coverage Report]

pass : 32 / 34 (94.12%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/planner/AbstractOlapTableScanNode.java 21 23 91.30% [86, 87]
🔵 com/starrocks/planner/OlapScanNode.java 4 4 100.00% []
🔵 com/starrocks/qe/ColocatedBackendSelector.java 7 7 100.00% []

@github-actions

Copy link
Copy Markdown
Contributor

[BE Incremental Coverage Report]

pass : 0 / 0 (0%)

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants