Skip to content

[BugFix] Avoid rewriting inside array_map lambda when pulling scan predicates up (backport #76380)#76396

Merged
wanpengfei-git merged 1 commit into
branch-4.0from
mergify/bp/branch-4.0/pr-76380
Jul 15, 2026
Merged

[BugFix] Avoid rewriting inside array_map lambda when pulling scan predicates up (backport #76380)#76396
wanpengfei-git merged 1 commit into
branch-4.0from
mergify/bp/branch-4.0/pr-76380

Conversation

@mergify

@mergify mergify Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

*** SIGSEGV (@0x58) received by PID 9963 (TID 0x149c001b06c0) LWP(10257) from PID 88; stack trace: ***
    @         0x32cb9b07 google::(anonymous namespace)::HandleSignal(int, siginfo_t*, void*)
    @     0x149c734a1ed3 (/usr/lib/x86_64-linux-gnu/libc.so.6+0xa1ed2)
    @         0x32cb9306 google::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*)
    @     0x149c73445330 (/usr/lib/x86_64-linux-gnu/libc.so.6+0x4532f)
    @         0x1d2e314d phmap::priv::raw_hash_set<phmap::priv::FlatHashMapPolicy<int, unsigned long>, starrocks::StdHash<int>, phmap
::EqualTo<int>, std::allocator<std::pair<int const, unsigned long> > >::end()
    @         0x1d2e3012 phmap::priv::raw_hash_set<phmap::priv::FlatHashMapPolicy<int, unsigned long>, starrocks::StdHash<int>, phmap
::EqualTo<int>, std::allocator<std::pair<int const, unsigned long> > >::end() const
    @         0x1d2e2ebf bool phmap::priv::raw_hash_set<phmap::priv::FlatHashMapPolicy<int, unsigned long>, starrocks::StdHash<int>,
phmap::EqualTo<int>, std::allocator<std::pair<int const, unsigned long> > >::contains<int>(int const&) const
    @         0x1d2e25f6 starrocks::Chunk::is_slot_exist(int) const
    @         0x1e1f0fba starrocks::Chunk::get_column_by_slot_id(int)
    @         0x2c8b8d8d starrocks::ColumnRef::get_column(starrocks::Expr*, starrocks::Chunk*)
    @         0x2c8b8d3a starrocks::ColumnRef::evaluate_checked(starrocks::ExprContext*, starrocks::Chunk*)
    @         0x2cac74ec starrocks::VectorizedInConstPredicate<(starrocks::LogicalType)7>::open(starrocks::RuntimeState*, starrocks::
ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @         0x2a4c3cf1 starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @         0x2a4c3cf1 starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @         0x2abba37d starrocks::ArrayMapExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @         0x2a4c3cf1 starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @         0x2ca28efc starrocks::VectorizedFunctionCallExpr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @         0x2a4c3cf1 starrocks::Expr::open(starrocks::RuntimeState*, starrocks::ExprContext*, starrocks::FunctionContext::FunctionStateScope)
    @         0x2a4bca6c starrocks::ExprContext::open(starrocks::RuntimeState*)
    @         0x2ca24de4 starrocks::ExprExecutor::open(std::vector<starrocks::ExprContext*, std::allocator<starrocks::ExprContext*> > const&, starrocks::RuntimeState*)
    @         0x1e6eaba3 starrocks::pipeline::SelectOperatorFactory::prepare(starrocks::RuntimeState*)
    @         0x23395101 starrocks::pipeline::Pipeline::prepare(starrocks::RuntimeState*)
    @         0x2338e36f starrocks::pipeline::NormalExecutionGroup::prepare_pipelines(starrocks::RuntimeState*)

A query using array_map with a lambda whose body contains a constant IN list crashes the BE (SIGSEGV) when enable_predicate_expr_reuse is on (the default). Minimal repro:

  WITH input AS (
      SELECT ARRAY_MAP(arg -> (CAST(MONTH(arg) AS BIGINT) IN (1, 2)), arr_datetime) AS arr
      FROM test_table
  )
  SELECT CAST(1 AS BIGINT) FROM input WHERE ARRAY_SUM(arr);
  Problematic plan (note 12: cast injected inside the array_map lambda's IN list):
  
  2:SELECT
  |  predicates: CAST(array_sum(array_map(<slot 10> -> CAST(month(<slot 10>) AS BIGINT) IN (12: cast, 2), 2: arr_datetime)) AS BOOLEAN)
  | 
  1:Project
  |  <slot 2> : 2: arr_datetime
  |  <slot 12> : 1                <-- outer SELECT CAST(1 AS BIGINT) becomes projection column slot 12
  |  
  0:OlapScanNode
     TABLE: test_table

12: cast is the outer SELECT CAST(1 AS BIGINT) output column. It is a valid column at the top of the SELECT node, but here it sits inside the lambda body, where only lambda arguments (<slot 10>) and captured columns exist — slot 12
is not in the lambda's local chunk.

Root cause: PullUpScanPredicateRule.replaceScalarOperator recursively rewrites every sub-expression of a pulled-up predicate to reuse scan projection columns, with no lambda boundary. When the outer query also selects a constant
(CAST(1 AS BIGINT)), that constant becomes a scan projection column (slot), and the structurally-identical constant sitting inside the array_map lambda's IN list gets rewritten into a ColumnRef of that column — producing ... IN
(, 2).

A lambda body is evaluated in a local chunk (only lambda arguments and captured columns exist), and the constant IN-predicate is built as VectorizedInConstPredicate on the BE, whose open() evaluates its elements against a null
chunk. Dereferencing that ColumnRef against the null chunk crashes the BE (Chunk::get_column_by_slot_id on nullptr).

What I'm doing:

Do not rewrite anything inside a LambdaFunctionOperator body in PullUpScanPredicateRule.replaceScalarOperator. Reuse of scan projection columns is only valid at the top level of the predicate, where the column exists in the
operator's input chunk; inside a lambda only lambda arguments and captured columns are in scope. This mirrors SubfieldCollector#visitLambdaFunctionOperator, which already never descends into lambdas. Lambda-internal
common-subexpression reuse continues to be handled correctly by ScalarOperatorsReuseRule, which is lambda-aware.

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5

This is an automatic backport of pull request #76380 done by [Mergify](https://mergify.com).

…edicates up (#76380)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit e3e1572)
@wanpengfei-git wanpengfei-git merged commit 6a70226 into branch-4.0 Jul 15, 2026
36 of 37 checks passed
@wanpengfei-git wanpengfei-git deleted the mergify/bp/branch-4.0/pr-76380 branch July 15, 2026 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants