Skip to content

[BugFix] Fix BE crash in primary key auto-increment partial update apply (backport #76119)#76175

Closed
mergify[bot] wants to merge 1 commit into
branch-3.5from
mergify/bp/branch-3.5/pr-76119
Closed

[BugFix] Fix BE crash in primary key auto-increment partial update apply (backport #76119)#76175
mergify[bot] wants to merge 1 commit into
branch-3.5from
mergify/bp/branch-3.5/pr-76119

Conversation

@mergify

@mergify mergify Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

For a primary-key table with an AUTO_INCREMENT column, a partial update that
omits that column applies through SegmentRewriter::rewrite_auto_increment, which
rebuilds the full segment from three sources: the keys / user-provided columns
re-read from the partial segment, the auto-increment column, and the missing
columns read from history.

rewrite_auto_increment swallowed the status of the partial-segment read: the
iterator was taken with StatusOr::value() (no ok() check) and get_next's status
was dropped (only a DCHECK guarded the row count, compiled out in release). A
transient read failure left read_chunk short, so the rebuilt segment recorded a
segment num_rows of 0 while its value columns still held N rows of real data.
The rowset meta and primary index kept the commit-time counts, so the
inconsistency stayed dormant until a later partial update of the same keys read
that segment via get_column_values, which silently skipped the 0-row segment,
returned an empty read column, and dereferenced a null source in
append_selective / SIMDGather (src=0x0) -- a crash that replays the pending
apply on every restart.

Shared-nothing only; the lake path (rewrite_auto_increment_lake)
already checks the read.

What I'm doing:

Root cause (write side), be/src/storage/rowset/segment_rewriter.cpp:

  • rewrite_auto_increment no longer swallows read errors: create the iterator
    with ASSIGN_OR_RETURN (an iterator-creation failure now returns cleanly
    instead of aborting in value()), propagate get_next's status, and always
    validate the read row count against the segment, with the expected/actual
    counts and segment path in the error. This mirrors rewrite_auto_increment_lake.
    Also closes a latent crash where itr->close() ran on a possibly-null iterator.

Read side (stops the crash on already-corrupt data),
be/src/storage/tablet_updates.cpp:

  • get_column_values returns Status::Corruption instead of silently skipping when
    a resolved rssid points to a 0-row segment (both the main read loop and the
    auto-increment default block). Corruption is non-retryable, so the tablet
    enters error state (recoverable via clone) rather than an endless crash-loop.
    Generalizes [BugFix] Guard against iterator UB in get_column_values when rssid not found #69617, which only guarded the rssid-underflow / out-of-range case.

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5

This is an automatic backport of pull request #76119 done by [Mergify](https://mergify.com).

…ply (#76119)

Signed-off-by: wyb <wybb86@gmail.com>
(cherry picked from commit ce03ba6)

# Conflicts:
#	be/src/storage/tablet_updates.cpp
#	be/test/storage/rowset/rowset_test.cpp
#	be/test/storage/tablet_updates_test.cpp
@mergify mergify Bot added the conflicts label Jul 10, 2026
@mergify

mergify Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Cherry-pick of ce03ba6 has failed:

On branch mergify/bp/branch-3.5/pr-76119
Your branch is up to date with 'origin/branch-3.5'.

You are currently cherry-picking commit ce03ba620f.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   be/src/storage/rowset/segment_rewriter.cpp

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   be/src/storage/tablet_updates.cpp
	both modified:   be/test/storage/rowset/rowset_test.cpp
	both modified:   be/test/storage/tablet_updates_test.cpp

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@wanpengfei-git wanpengfei-git enabled auto-merge (squash) July 10, 2026 06:10
@mergify mergify Bot closed this Jul 10, 2026
auto-merge was automatically disabled July 10, 2026 06:10

Pull request was closed

@mergify

mergify Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@mergify[bot]: Backport conflict, please reslove the conflict and resubmit the pr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant