Skip to content

Commit b7dc4eb

Browse files
Youngwbclaude
Youngwb
and
claude
committed
[Refactor] Remove IVM FULL refresh mode (StarRocks#72288)
Signed-off-by: Youngwb <yangwenbo_mailbox@163.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent a9b883c commit b7dc4eb

11 files changed

Lines changed: 4 additions & 541 deletions

File tree

‎docs/en/sql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -407,7 +407,6 @@ Properties of the asynchronous materialized view. You can modify the properties
407407

408408
- `PCT`: (Default) For partitioned materialized views, only the affected partition is refreshed when there is a data change, ensuring result consistency for that partition. For non-partitioned materialized views, any data change in the base table triggers a full refresh of the materialized view.
409409
- `INCREMENTAL`: Ensures that only incremental refreshes are performed. If the materialized view does not support incremental refresh based on its definition or encounters non-incremental data, creation or refresh will fail.
410-
- `FULL`: Forces a full refresh of all data every time, regardless of whether the materialized view supports incremental or partition-level refresh.
411410

412411
<MVWarehouse />
413412

‎docs/ja/sql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -410,7 +410,6 @@ ALTER MATERIALIZED VIEW <mv_name> SET ("bloom_filter_columns" = "");
410410

411411
- `PCT`: （デフォルト）パーティション化されたマテリアライズドビューの場合、ベーステーブルにデータの変更があると影響を受けたパーティションのみリフレッシュされ、そのパーティションの結果の一貫性が保証されます。パーティション化されていないマテリアライズドビューの場合、ベーステーブルのいずれかが変更されるとマテリアライズドビュー全体がフルリフレッシュされます。
412412
- `INCREMENTAL`: 増分リフレッシュのみを行うことを保証します。マテリアライズドビューの定義で増分リフレッシュがサポートされていない場合や、非増分データに遭遇した場合、作成やリフレッシュが失敗します。
413-
- `FULL`: マテリアライズドビューが増分やパーティション単位のリフレッシュをサポートしているかどうかに関係なく、毎回全データのフルリフレッシュを強制します。
414413

415414
<MVWarehouse />
416415

‎docs/zh/sql-reference/sql-statements/materialized_view/CREATE_MATERIALIZED_VIEW.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -404,7 +404,6 @@ ALTER MATERIALIZED VIEW <mv_name> SET ("bloom_filter_columns" = "");
404404

405405
- `PCT`：（默认）对于分区物化视图，当基表数据发生变化时，仅刷新受影响的分区，保证该分区的数据一致性。对于非分区物化视图，基表任何数据变化都会触发全量刷新。
406406
- `INCREMENTAL`：仅允许进行增量刷新。如果根据定义物化视图不支持增量刷新，或遇到无法增量处理的数据，则创建或刷新的操作会失败。
407-
- `FULL`：每次都强制进行全量刷新，无论物化视图是否支持增量刷新或分区级刷新。
408407

409408
<MVWarehouse />
410409

‎fe/fe-core/src/main/java/com/starrocks/catalog/MaterializedView.java‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -194,10 +194,6 @@ public enum RefreshMode {
194194
* the last refresh.
195195
*/
196196
PCT,
197-
/**
198-
* FULL: Full refresh mode, refresh all partitions of the materialized view.
199-
*/
200-
FULL,
201197
/**
202198
* INCREMENTAL: Incremental refresh mode, only refresh the incremental changed rows since the last refresh.
203199
*/
@@ -215,10 +211,6 @@ public boolean isIncremental() {
215211
return this == INCREMENTAL;
216212
}
217213

218-
public boolean isFull() {
219-
return this == FULL;
220-
}
221-
222214
public boolean isIncrementalOrAuto() {
223215
return this == INCREMENTAL || this == AUTO;
224216
}

‎fe/fe-core/src/main/java/com/starrocks/scheduler/mv/MVRefreshProcessor.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -640,7 +640,7 @@ public void refreshExternalTable(Map<BaseTableSnapshotInfo, PCellSortedSet> base
640640
if (currentRefreshMode.isIncremental()) {
641641
throw new SemanticException("Materialized view %s.%s refresh failed: base table %s schema " +
642642
"or identity changed, cannot do incremental refresh in %s mode. " +
643-
"Please trigger a full refresh.",
643+
"Please drop and re-create the materialized view.",
644644
db.getFullName(), mv.getName(), baseTableInfo.getTableInfoStr(), currentRefreshMode);
645645
}
646646
toRepairTables.add(Pair.create(newTable, baseTableInfo));

‎fe/fe-core/src/main/java/com/starrocks/scheduler/mv/MVRefreshProcessorFactory.java‎

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,13 +18,10 @@
1818
import com.starrocks.catalog.MaterializedView;
1919
import com.starrocks.metric.IMaterializedViewMetricsEntity;
2020
import com.starrocks.scheduler.MvTaskRunContext;
21-
import com.starrocks.scheduler.TaskRun;
2221
import com.starrocks.scheduler.mv.hybrid.MVHybridRefreshProcessor;
2322
import com.starrocks.scheduler.mv.ivm.MVIVMRefreshProcessor;
2423
import com.starrocks.scheduler.mv.pct.MVPCTRefreshProcessor;
2524

26-
import java.util.Map;
27-
2825
public class MVRefreshProcessorFactory {
2926
public static final MVRefreshProcessorFactory INSTANCE = new MVRefreshProcessorFactory();
3027

@@ -37,12 +34,6 @@ public MVRefreshProcessor newProcessor(Database db, MaterializedView mv,
3734
return new MVIVMRefreshProcessor(db, mv, mvContext, mvEntity, refreshMode);
3835
case AUTO:
3936
return new MVHybridRefreshProcessor(db, mv, mvContext, mvEntity, refreshMode);
40-
case FULL: {
41-
// full refresh
42-
Map<String, String> props = mvContext.getProperties();
43-
props.put(TaskRun.FORCE, "true");
44-
return new MVPCTRefreshProcessor(db, mv, mvContext, mvEntity, refreshMode);
45-
}
4637
default:
4738
return new MVPCTRefreshProcessor(db, mv, mvContext, mvEntity, refreshMode);
4839
}

‎fe/fe-core/src/test/java/com/starrocks/alter/AlterMVJobExecutorEditLogTest.java‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -450,9 +450,9 @@ public void testVisitModifyTablePropertiesClausePartitionRefreshStrategyNormalCa
450450

451451
@Test
452452
public void testVisitModifyTablePropertiesClauseMvRefreshModeNormalCase() throws Exception {
453-
// Test PROPERTIES_MV_REFRESH_MODE - valid values: AUTO, PCT, FULL, INCREMENTAL
454-
// Use FULL as it doesn't have additional constraints like AUTO/INCREMENTAL
455-
testPropertyNormalCase(PropertyAnalyzer.PROPERTIES_MV_REFRESH_MODE, "FULL");
453+
// Test PROPERTIES_MV_REFRESH_MODE - valid values: AUTO, PCT, INCREMENTAL
454+
// Use PCT as it doesn't have additional constraints like AUTO/INCREMENTAL
455+
testPropertyNormalCase(PropertyAnalyzer.PROPERTIES_MV_REFRESH_MODE, "PCT");
456456
}
457457

458458
@Test

‎fe/fe-core/src/test/java/com/starrocks/alter/AlterMaterializedViewTest.java‎

Lines changed: 0 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -66,12 +66,6 @@ public void testChangeMVRefreshMode1() throws Exception {
6666
alterMaterializedView(alterStmt, false);
6767
Assertions.assertEquals(MaterializedView.RefreshMode.AUTO, mv.getCurrentRefreshMode());
6868
}
69-
// change refresh mode from auto to full
70-
{
71-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"full\")";
72-
alterMaterializedView(alterStmt, false);
73-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
74-
}
7569
}
7670

7771
@Test
@@ -92,12 +86,6 @@ public void testChangeMVRefreshMode2() throws Exception {
9286
alterMaterializedView(alterStmt, true);
9387
Assertions.assertEquals(MaterializedView.RefreshMode.PCT, mv.getCurrentRefreshMode());
9488
}
95-
// change refresh mode to full
96-
{
97-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"full\")";
98-
alterMaterializedView(alterStmt, false);
99-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
100-
}
10189
}
10290

10391
@Test
@@ -118,64 +106,6 @@ public void testChangeMVRefreshMode3() throws Exception {
118106
alterMaterializedView(alterStmt, false);
119107
Assertions.assertEquals(MaterializedView.RefreshMode.AUTO, mv.getCurrentRefreshMode());
120108
}
121-
// change refresh mode from auto to full
122-
{
123-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"full\")";
124-
alterMaterializedView(alterStmt, false);
125-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
126-
}
127-
}
128-
129-
@Test
130-
public void testChangeMVRefreshMode4() throws Exception {
131-
String query = "SELECT id, data, date FROM `iceberg0`.`unpartitioned_db`.`t0` as a;";
132-
MaterializedView mv = createMaterializedViewWithRefreshMode(query, "full");
133-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
134-
135-
// change refresh mode from auto to incremental
136-
{
137-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"incremental\")";
138-
alterMaterializedView(alterStmt, true);
139-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
140-
}
141-
// change refresh mode from incremental to auto
142-
{
143-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"auto\")";
144-
alterMaterializedView(alterStmt, true);
145-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
146-
}
147-
// change refresh mode from auto to full
148-
{
149-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"full\")";
150-
alterMaterializedView(alterStmt, false);
151-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
152-
}
153-
}
154-
155-
@Test
156-
public void testChangeMVRefreshMode5() throws Exception {
157-
String query = "SELECT id, count(data) over (partition by date) FROM `iceberg0`.`unpartitioned_db`.`t0` as a;";
158-
MaterializedView mv = createMaterializedViewWithRefreshMode(query, "full");
159-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
160-
161-
// change refresh mode from auto to incremental
162-
{
163-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"incremental\")";
164-
alterMaterializedView(alterStmt, true);
165-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
166-
}
167-
// change refresh mode to auto
168-
{
169-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"auto\")";
170-
alterMaterializedView(alterStmt, true);
171-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
172-
}
173-
// change refresh mode to full
174-
{
175-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"full\")";
176-
alterMaterializedView(alterStmt, false);
177-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
178-
}
179109
}
180110

181111
@Test
@@ -196,12 +126,6 @@ public void testChangeMVRefreshMode6() throws Exception {
196126
alterMaterializedView(alterStmt, true);
197127
Assertions.assertEquals(MaterializedView.RefreshMode.PCT, mv.getCurrentRefreshMode());
198128
}
199-
// change refresh mode from auto to full
200-
{
201-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"full\")";
202-
alterMaterializedView(alterStmt, false);
203-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
204-
}
205129
}
206130

207131
@Test
@@ -222,12 +146,6 @@ public void testChangeMVRefreshMode7() throws Exception {
222146
alterMaterializedView(alterStmt, true);
223147
Assertions.assertEquals(MaterializedView.RefreshMode.PCT, mv.getCurrentRefreshMode());
224148
}
225-
// change refresh mode to full
226-
{
227-
String alterStmt = "alter materialized view test_mv1 set (\"refresh_mode\" = \"full\")";
228-
alterMaterializedView(alterStmt, false);
229-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
230-
}
231149
}
232150

233151
@Test

‎fe/fe-core/src/test/java/com/starrocks/scheduler/mv/ivm/IVMBasedMvRefreshProcessorIcebergTest.java‎

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -419,20 +419,6 @@ public void testIncrementalRefreshWithScanOperator() throws Exception {
419419
Assertions.assertEquals(MaterializedView.RefreshMode.INCREMENTAL, mv.getCurrentRefreshMode());
420420
}
421421

422-
@Test
423-
public void testFullRefreshWithScanOperator() throws Exception {
424-
String query = "SELECT id, data, date FROM `iceberg0`.`unpartitioned_db`.`t0` as a;";
425-
MaterializedView mv = createMaterializedViewWithRefreshMode(query, "full");
426-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
427-
}
428-
429-
@Test
430-
public void testFullRefreshWithTableFunctionOperator() throws Exception {
431-
String query = "SELECT id, unnest FROM `iceberg0`.`unpartitioned_db`.`t0` as a, unnest(split(data, ','));";
432-
MaterializedView mv = createMaterializedViewWithRefreshMode(query, "full");
433-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
434-
}
435-
436422
@Test
437423
public void testAutoRefreshWithScanOperator() throws Exception {
438424
String query = "SELECT id, data, date FROM `iceberg0`.`unpartitioned_db`.`t0` as a;";
@@ -448,13 +434,6 @@ public void testIncrementalRefreshWithOlapScanOperator() throws Exception {
448434
});
449435
}
450436

451-
@Test
452-
public void testFullRefreshWithOlapScanOperator() throws Exception {
453-
String query = "SELECT * from emps;";
454-
MaterializedView mv = createMaterializedViewWithRefreshMode(query, "full");
455-
Assertions.assertEquals(MaterializedView.RefreshMode.FULL, mv.getCurrentRefreshMode());
456-
}
457-
458437
@Test
459438
public void testAutoRefreshWithOlapScanOperator() throws Exception {
460439
String query = "SELECT * from emps;";

0 commit comments

Comments
 (0)