[BugFix] Cover projected sstable max_rss_rowid in update_next_rowset_id (backport #72172)

[mergify]

Why I'm doing:

Repeated SPLIT/MERGE cycles on a shared-data range-distribution PK table get stuck because a background COMPACTION publish on a SPLIT child fails with "sstables are not ordered" — and the next reshard job parks in PREPARING waiting for the publish to finish.

What I'm doing:

merge_sstables (be/src/storage/lake/tablet_merger.cpp:615-618) projects each child's sstable max_rss_rowid by adding ctx.rssid_offset to the high word. The legacy non-shared_rssid branch can produce a projected high that exceeds every rowset.id in the merged metadata — for example, a delete-only sstable whose memtable max_rss_rowid was set to (rowset_id<<32|UINT32_MAX) (be/src/storage/lake/persistent_index_memtable.cpp:110, 131) contributes a high rssid that has no surviving rowset to pin it.

update_next_rowset_id walked only metadata->rowsets(), so the merged tablet's next_rowset_id was set from rowset.id alone. Any subsequent write on this tablet — and on every SPLIT child that inherits this metadata via std::make_shared<TabletMetadataPB>(*old_tablet_metadata) in split_tablet (be/src/storage/lake/tablet_splitter.cpp:486) — would assign a new rssid less than the projected sstable highs, producing a sstable whose max_rss_rowid falls below an existing entry in sstable_meta. That breaks the ascending-order invariant LakePersistentIndex::commit() enforces (be/src/storage/lake/lake_persistent_index.cpp:881):

sstables are not ordered, last_max_rss_rowid=A : max_rss_rowid=B

The COMPACTION transaction whose publish triggered the check parks in COMMITTED. The next reshard job on the same tablet allocates its commitVersion via getNextTransactionId() and waits in PREPARING for visibleVersion to catch up (MergeTabletJob.runPreparingJob), but the stuck compaction blocks the publish chain — so PREPARING hangs indefinitely and the table is parked in TABLET_RESHARD.

Reproducer (shared-data PK range distribution, 3-CN cluster)

1. Seed N rows on a range-distributed PK table.
2. Run repeated SPLIT/MERGE cycles, each with overlapping UPSERTs:
   SPLIT 1->8, UPSERT, MERGE 8->1, SPLIT 1->8, UPSERT, MERGE 8->1, ...
3. After the second MERGE, a background COMPACTION on a SPLIT child fails to publish:

   Fail to publish version for tablets:[[27673, 27666, 27672, 27667, 27668, 27670, 27671, 27669]],
   error msg: sstables are not ordered,
   last_max_rss_rowid=1090921695183 : max_rss_rowid=601295447439
   

   (high words 254 vs 140; the new compaction output sstable's projected high is below an inherited shared sstable's projected high.)
4. The next MERGE_TABLET job sits in JOB_STATE=PREPARING; SHOW PROC "/transactions/<db>/running" shows the COMPACTION_… txn parked in COMMITTED, and the table state stays in TABLET_RESHARD.

Fix

In update_next_rowset_id also walk sstable_meta.sstables(*) and bound next_rowset_id by (max projected high) + 1, clamping at UINT32_MAX to match the rssid encoding. Future writes — including those on any SPLIT child inheriting this metadata — are guaranteed to assign rssids strictly greater than every projected sstable high, so new sstables append to sstable_meta in ascending max_rss_rowid order.

Fixes #64986

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
  • This pr needs auto generate documentation
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 4.1
  • 4.0
  • 3.5
  • 3.4

    ---

    This is an automatic backport of pull request [BugFix] Cover projected sstable max_rss_rowid in update_next_rowset_id #72172 done by Mergify.