Skip to content

[Refactor] Split data sinks into dedicated modules#76312

Merged
alvin-phoenix-ai merged 2 commits into
StarRocks:mainfrom
alvin-phoenix-ai:opt-ds-02
Jul 14, 2026
Merged

[Refactor] Split data sinks into dedicated modules#76312
alvin-phoenix-ai merged 2 commits into
StarRocks:mainfrom
alvin-phoenix-ai:opt-ds-02

Conversation

@alvin-phoenix-ai

Copy link
Copy Markdown
Contributor

Why I'm doing:

What I'm doing:

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5

Signed-off-by: alvin-phoenix-ai <alvin.zhao@phoenixdata.ai>
@CelerData-Reviewer

Copy link
Copy Markdown

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 988c562e84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread be/src/exec/CMakeLists.txt
Signed-off-by: alvin-phoenix-ai <alvin.zhao@phoenixdata.ai>
@github-actions

Copy link
Copy Markdown
Contributor

[Java-Extensions Incremental Coverage Report]

pass : 0 / 0 (0%)

@github-actions

Copy link
Copy Markdown
Contributor

[FE Incremental Coverage Report]

pass : 0 / 0 (0%)

@alvin-phoenix-ai
alvin-phoenix-ai merged commit f2940f3 into StarRocks:main Jul 14, 2026
93 of 98 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

[BE Incremental Coverage Report]

fail : 2874 / 4484 (64.09%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/data_sink/tablet/tablet_sink_colocate_sender.h 0 1 00.00% [30]
🔵 be/src/data_sink/external/mysql_table_sink.h 0 2 00.00% [72, 74]
🔵 be/src/data_sink/result/memory_scratch_sink.h 0 2 00.00% [83, 88]
🔵 be/src/data_sink/external/mysql_table_sink.cpp 0 33 00.00% [55, 56, 58, 60, 61, 63, 65, 66, 67, 68, 69, 70, 71, 74, 75, 81, 82, 84, 85, 86, 88, 89, 90, 92, 95, 97, 98, 99, 105, 107, 113, 114, 115]
🔵 be/src/data_sink/external/mysql_table_writer.cpp 0 103 00.00% [60, 61, 63, 64, 65, 67, 68, 70, 71, 72, 74, 76, 78, 79, 80, 83, 84, 87, 88, 92, 93, 94, 97, 99, 101, 106, 108, 110, 112, 115, 116, 117, 119, 121, 122, 123, 124, 125, 128, 131, 134, 135, 136, 137, 139, 140, 141, 143, 144, 145, 146, 148, 150, 151, 152, 156, 158, 160, 161, 162, 165, 166, 168, 170, 171, 173, 175, 176, 177, 179, 181, 183, 185, 188, 190, 191, 194, 197, 198, 201, 202, 203, 207, 208, 209, 210, 212, 214, 215, 216, 217, 218, 219, 220, 221, 223, 226, 227, 229, 230, 231, 234, 235]
🔵 be/src/data_sink/result/customized_result_writer.cpp 0 76 00.00% [30, 32, 33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 46, 49, 50, 51, 52, 54, 56, 57, 59, 60, 61, 64, 65, 67, 68, 70, 71, 72, 73, 75, 76, 78, 79, 81, 82, 83, 86, 88, 89, 91, 92, 93, 94, 95, 96, 97, 99, 100, 101, 104, 105, 106, 107, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132]
🔵 be/src/data_sink/external/export_sink.h 0 2 00.00% [73, 75]
🔵 be/src/data_sink/tablet/range_tablet_sink_sender.h 0 4 00.00% [25, 30, 32, 34]
🔵 be/src/data_sink/tablet/tablet_sink_colocate_sender.cpp 0 215 00.00% [28, 32, 35, 37, 43, 44, 45, 48, 49, 50, 51, 52, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 69, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 83, 85, 87, 90, 92, 93, 94, 95, 96, 98, 99, 100, 102, 103, 105, 107, 108, 109, 110, 111, 112, 113, 118, 119, 124, 125, 126, 127, 128, 133, 134, 136, 138, 139, 140, 141, 142, 144, 145, 148, 149, 150, 152, 153, 154, 157, 158, 159, 160, 163, 164, 165, 167, 168, 169, 170, 173, 176, 177, 178, 180, 181, 182, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 198, 199, 200, 202, 204, 205, 206, 209, 210, 211, 213, 214, 215, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 228, 229, 230, 232, 234, 235, 237, 239, 241, 242, 243, 245, 246, 247, 248, 251, 254, 256, 257, 259, 261, 262, 263, 264, 265, 266, 268, 270, 271, 272, 273, 274, 275, 276, 278, 279, 280, 281, 282, 284, 285, 286, 287, 289, 290, 291, 293, 295, 296, 297, 298, 299, 302, 303, 304, 305, 306, 308, 309, 311, 312, 313, 314, 316, 317, 318, 320, 321, 322, 324, 325, 326, 328, 329, 331, 332, 333, 334, 335, 336, 337, 339, 340]
🔵 be/src/data_sink/external/export_sink.cpp 0 90 00.00% [54, 55, 57, 59, 60, 61, 64, 65, 68, 69, 71, 73, 74, 76, 77, 80, 83, 84, 85, 87, 88, 90, 92, 94, 95, 96, 97, 100, 101, 102, 104, 105, 106, 107, 108, 109, 110, 111, 112, 115, 116, 117, 118, 119, 120, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 138, 139, 140, 141, 142, 144, 145, 148, 149, 150, 151, 153, 154, 155, 158, 159, 160, 163, 166, 167, 168, 169, 171, 172, 173, 174, 175, 176, 177, 178]
🔵 be/src/data_sink/external/blackhole_table_sink.cpp 0 7 00.00% [21, 22, 23, 24, 25, 26, 27]
🔵 be/src/data_sink/external/mysql_table_writer.h 0 1 00.00% [55]
🔵 be/src/data_sink/exchange/data_stream_sender.cpp 49 356 13.76% [99, 133, 135, 138, 139, 140, 141, 143, 144, 145, 146, 147, 148, 150, 151, 196, 197, 198, 201, 202, 203, 204, 206, 207, 208, 209, 210, 211, 212, 215, 216, 217, 218, 219, 220, 222, 223, 224, 225, 227, 228, 230, 232, 233, 234, 237, 238, 240, 243, 244, 247, 248, 249, 250, 255, 261, 262, 264, 265, 266, 268, 269, 270, 271, 273, 276, 277, 278, 279, 280, 281, 282, 283, 284, 286, 287, 289, 290, 291, 292, 294, 295, 296, 297, 298, 299, 300, 301, 303, 306, 307, 310, 313, 314, 317, 318, 321, 322, 323, 326, 327, 329, 332, 333, 334, 337, 338, 339, 340, 342, 343, 347, 350, 351, 352, 353, 354, 355, 356, 357, 358, 361, 362, 363, 364, 419, 427, 428, 429, 430, 433, 434, 435, 438, 440, 442, 443, 444, 445, 447, 448, 451, 452, 453, 454, 455, 457, 459, 463, 464, 465, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 478, 479, 482, 483, 488, 490, 492, 493, 501, 503, 504, 505, 508, 509, 510, 511, 512, 515, 518, 520, 521, 523, 524, 525, 526, 527, 529, 530, 531, 532, 536, 537, 538, 539, 540, 542, 544, 546, 548, 549, 550, 551, 552, 553, 555, 556, 557, 558, 563, 564, 565, 570, 572, 573, 574, 575, 580, 581, 584, 585, 586, 588, 590, 591, 592, 593, 595, 597, 599, 601, 603, 604, 607, 608, 610, 611, 612, 618, 619, 620, 621, 622, 623, 625, 626, 632, 633, 635, 637, 638, 640, 641, 643, 644, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 662, 664, 665, 666, 670, 671, 672, 673, 675, 676, 677, 678, 681, 683, 684, 687, 689, 690, 691, 694, 695, 696, 697, 700, 701, 702, 703, 705, 706, 707, 710, 711, 712, 713, 714, 715, 717]
🔵 be/src/data_sink/external/noop_sink.h 2 11 18.18% [26, 27, 28, 29, 30, 31, 32, 34, 36]
🔵 be/src/data_sink/result/result_sink.cpp 15 77 19.48% [86, 88, 90, 91, 94, 95, 96, 97, 98, 100, 103, 105, 108, 109, 113, 114, 115, 116, 119, 120, 121, 124, 125, 127, 128, 130, 131, 132, 133, 136, 137, 138, 140, 141, 142, 145, 151, 152, 153, 155, 156, 157, 160, 162, 163, 164, 166, 168, 171, 172, 173, 175, 177, 178, 179, 180, 182, 183, 184, 186, 187, 188]
🔵 be/src/data_sink/tablet/tablet_sink_sender.h 5 17 29.41% [59, 60, 61, 63, 76, 77, 78, 79, 80, 81, 82, 84]
🔵 be/src/data_sink/exchange/multi_cast_data_stream_sink.cpp 8 26 30.77% [34, 35, 38, 39, 42, 43, 46, 47, 50, 51, 52, 53, 54, 55, 57, 58, 60, 61]
🔵 be/src/data_sink/exchange/multi_cast_data_stream_sink.h 2 6 33.33% [32, 42, 43, 47]
🔵 be/src/data_sink/result/result_writer.h 3 9 33.33% [61, 69, 70, 71, 82, 83]
🔵 be/src/data_sink/external/blackhole_table_sink.h 2 4 50.00% [27, 29]
🔵 be/src/data_sink/external/schema_table_sink.cpp 52 91 57.14% [70, 72, 73, 74, 76, 77, 78, 79, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 93, 94, 95, 96, 97, 99, 100, 101, 102, 103, 104, 106, 107, 112, 122, 123, 128, 129, 134, 151]
🔵 be/src/data_sink/result/mysql_result_writer.cpp 106 160 66.25% [166, 173, 184, 185, 186, 188, 189, 190, 191, 194, 195, 196, 199, 200, 201, 202, 204, 207, 208, 209, 211, 212, 213, 214, 215, 216, 217, 219, 220, 221, 222, 223, 224, 226, 227, 228, 230, 231, 232, 233, 235, 236, 237, 238, 244, 245, 248, 249, 250, 285, 286, 288, 289, 290]
🔵 be/src/data_sink/result/statistic_result_writer.cpp 324 457 70.90% [54, 64, 65, 66, 67, 68, 70, 72, 73, 75, 76, 77, 80, 81, 82, 99, 123, 128, 143, 149, 158, 207, 209, 212, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 226, 227, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 243, 244, 246, 247, 248, 250, 251, 400, 402, 405, 408, 409, 410, 411, 412, 413, 414, 415, 417, 418, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 432, 433, 435, 436, 437, 439, 440, 486, 496, 499, 502, 503, 504, 505, 506, 508, 509, 510, 511, 512, 515, 516, 518, 519, 520, 522, 523, 625, 627, 630, 632, 633, 634, 635, 636, 637, 638, 640, 641, 643, 644, 645, 646, 647, 648, 649, 650, 651, 654, 655, 657, 658, 659, 661, 662]
🔵 be/src/data_sink/result/metadata_result_writer.cpp 87 111 78.38% [40, 45, 46, 47, 48, 49, 51, 53, 54, 56, 57, 58, 61, 62, 63, 80, 95, 180, 181, 182, 183, 185, 186, 197]
🔵 be/src/data_sink/tablet/multi_olap_table_sink.cpp 61 76 80.26% [41, 42, 43, 45, 101, 102, 103, 105, 124, 125, 126, 127, 128, 130, 133]
🔵 be/src/data_sink/tablet/tablet_sink_index_channel.cpp 700 867 80.74% [61, 101, 117, 118, 119, 121, 146, 147, 148, 154, 155, 156, 157, 158, 159, 193, 194, 195, 196, 210, 255, 256, 257, 262, 263, 282, 317, 318, 319, 320, 321, 322, 323, 324, 326, 328, 329, 330, 345, 385, 389, 397, 398, 401, 402, 403, 404, 406, 407, 452, 459, 460, 461, 471, 472, 473, 474, 479, 480, 481, 483, 484, 485, 486, 489, 491, 492, 495, 497, 498, 499, 502, 503, 504, 505, 508, 509, 540, 589, 595, 597, 598, 601, 602, 603, 606, 609, 612, 615, 617, 618, 619, 623, 624, 626, 627, 632, 633, 637, 639, 642, 643, 760, 761, 762, 763, 793, 794, 795, 796, 797, 798, 799, 800, 802, 803, 804, 805, 816, 817, 818, 819, 820, 821, 822, 823, 824, 826, 827, 828, 829, 831, 834, 835, 836, 847, 862, 894, 954, 971, 972, 983, 1037, 1038, 1039, 1060, 1061, 1072, 1073, 1093, 1094, 1095, 1240, 1241, 1242, 1250, 1251, 1266, 1267, 1268, 1280, 1281, 1318, 1319, 1320, 1345, 1373]
🔵 be/src/data_sink/tablet/tablet_sink.cpp 626 747 83.80% [174, 244, 245, 246, 247, 272, 273, 277, 278, 279, 284, 285, 286, 287, 288, 289, 290, 330, 334, 335, 336, 337, 339, 340, 341, 342, 368, 369, 370, 372, 380, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 401, 402, 403, 415, 416, 468, 469, 470, 477, 570, 571, 572, 574, 580, 599, 600, 601, 602, 615, 616, 617, 618, 619, 624, 625, 657, 658, 674, 685, 761, 762, 834, 835, 836, 850, 856, 871, 872, 873, 874, 875, 876, 915, 927, 931, 932, 944, 955, 983, 984, 985, 986, 987, 988, 990, 1019, 1020, 1021, 1022, 1023, 1024, 1026, 1028, 1029, 1033, 1034, 1035, 1039, 1064, 1096, 1108, 1109, 1110, 1111, 1113, 1134, 1135, 1136, 1137, 1139]
🔵 be/src/data_sink/tablet/tablet_sink_sender.cpp 214 249 85.94% [104, 136, 137, 138, 139, 140, 142, 143, 147, 214, 215, 216, 217, 220, 223, 246, 247, 248, 249, 252, 255, 329, 330, 331, 333, 335, 336, 339, 399, 400, 401, 404, 405, 406, 408]
🔵 be/src/data_sink/result/http_result_writer.cpp 61 71 85.92% [41, 64, 65, 102, 103, 108, 109, 111, 112, 113]
🔵 be/src/data_sink/result/file_result_writer.cpp 95 111 85.59% [83, 84, 85, 93, 94, 116, 126, 127, 155, 156, 179, 180, 181, 182, 183, 196]
🔵 be/src/data_sink/result/result_sink.h 8 9 88.89% [70]
🔵 be/src/data_sink/exchange/data_stream_sender.h 8 9 88.89% [108]
🔵 be/src/data_sink/result/memory_scratch_sink.cpp 46 52 88.46% [108, 117, 118, 123, 134, 135]
🔵 be/src/data_sink/tablet/range_router.cpp 147 164 89.63% [35, 38, 39, 51, 94, 95, 96, 106, 131, 168, 172, 175, 192, 193, 255, 256, 267]
🔵 be/src/data_sink/result/arrow_result_writer.cpp 28 31 90.32% [63, 81, 82]
🔵 be/src/data_sink/tablet/tablet_sink_index_channel.h 59 63 93.65% [201, 202, 247, 248]
🔵 be/src/data_sink/tablet/range_tablet_sink_sender.cpp 76 81 93.83% [44, 111, 112, 114, 143]
🔵 be/src/data_sink/result/file_result_writer.h 25 26 96.15% [72]
🔵 be/src/data_sink/result/buffer_control_result_writer.cpp 27 28 96.43% [41]
🔵 be/src/data_sink/result/mysql_column_writer.h 27 28 96.43% [90]
🔵 be/src/data_sink/result/arrow_result_writer.h 1 1 100.00% []
🔵 be/src/data_sink/result/buffer_control_result_writer.h 1 1 100.00% []
🔵 be/src/data_sink/result/http_result_writer.h 1 1 100.00% []
🔵 be/src/data_sink/tablet/range_router.h 2 2 100.00% []
🔵 be/src/data_sink/tablet/tablet_sink.h 4 4 100.00% []
🔵 be/src/data_sink/tablet/multi_olap_table_sink.h 1 1 100.00% []
🔵 be/src/data_sink/external/schema_table_sink.h 1 1 100.00% []

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants