Skip to content

[BugFix] FSE-v2 missed setting the schema for shared-data sorted schema change (backport #72235)#72378

Merged
xiangguangyxg merged 1 commit into
StarRocks:branch-4.0from
banmoy:backport-72235-branch-4.0
Apr 30, 2026
Merged

[BugFix] FSE-v2 missed setting the schema for shared-data sorted schema change (backport #72235)#72378
xiangguangyxg merged 1 commit into
StarRocks:branch-4.0from
banmoy:backport-72235-branch-4.0

Conversation

@banmoy

@banmoy banmoy commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Why I'm doing:

On shared-data, an ALTER that changes the sort key is wrongly cancelled
with a misleading "table not found" when the schema cache is disabled:

update information_schema.be_configs
    set value = "0" where name = "lake_metadata_cache_limit";

ALTER TABLE t1 ADD COLUMN k0 INT KEY DEFAULT "0" FIRST;
W table_schema_service.cpp:333 failed to get schema.
  db_id: 0, table_id: 0, schema_id: 10422, tablet_id: 10423, txn_id: 249,
  error: Table not exist: table not found, may be dropped, db id: 0, table id: 0

Walking the cause:

  • This alter runs through SortedSchemaChange, which writes the converted
    rowsets through DeltaWriter. The builder is only fed schema_id; the
    writer is left to resolve the schema itself.

  • Since [Enhancement] Add BE TableSchemaService for Fast Schema Evolution in shared-data #66699 (FSE-v2 in shared-data, merged Dec 2025), DeltaWriter
    resolves the schema through TableSchemaService, whose lookup chain is:

    schema cache (by schema_id)
       -> latest cached tablet metadata
       -> FE RPC (by db_id, table_id, schema_id)
    
  • Shared-data alter tasks (TAlterTabletReqV2) don't carry catalog ids.
    When the lookup falls through to the RPC, the request goes out with
    db_id=0, table_id=0 and FE rightly answers "Table not exist".

The bug has been latent because the cache (default 2 GB) and cached
tablet metadata almost always answer the schema lookup before the RPC.
Disabling the cache exposes the broken RPC path.

DirectSchemaChange is unaffected — it builds its writer with a
TabletSchemaPtr directly and never touches the service.

What I'm doing:

The sorted schema change already holds the new tablet schema; the
schema-service lookup is unnecessary. Hand the schema to DeltaWriter
directly.

  • DeltaWriterBuilder gains set_tablet_schema(TabletSchemaPtr). When
    set, DeltaWriter::init_tablet_schema() short-circuits and bypasses
    TableSchemaService entirely. set_schema_id is still required and
    validated against the preset schema's id.
  • SortedSchemaChange::process passes the new tablet schema into the
    builder. DirectSchemaChange is unchanged.

As a side effect this also removes one (normally successful) RPC per
sorted schema change in the cache-hit case — the residual TODO already
flagged in schema_change.cpp.

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
    • BE UT: a regression test that disables the metacache, hooks the
      schema RPC sync point, and asserts the sorted schema change runs
      end-to-end without firing the RPC.
    • SQL: test_fse_alter_with_metacache_off @cloud pinning the original
      repro.
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5
    • 3.4

…ma change (backport StarRocks#72235)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: wanpengfei-git <wanpengfei91@163.com>
(cherry picked from commit d70921f)

Signed-off-by: PengFei Li <lpengfei2016@gmail.com>
@mergify mergify Bot assigned banmoy Apr 30, 2026
@xiangguangyxg xiangguangyxg merged commit d3d1dda into StarRocks:branch-4.0 Apr 30, 2026
55 of 56 checks passed
@banmoy banmoy deleted the backport-72235-branch-4.0 branch April 30, 2026 06:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants