starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Feature] Support rollup of lake table

Open smartlxh opened this issue 1 year ago • 3 comments

Why I'm doing:

What I'm doing:

Fixes #issue

What type of PR is this:

  • [ ] BugFix
  • [x] Feature
  • [ ] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [x] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [ ] 3.3
    • [ ] 3.2
    • [ ] 3.1
    • [ ] 3.0
    • [ ] 2.5

smartlxh avatar Apr 19 '24 03:04 smartlxh

Quality Gate Failed Quality Gate failed

Failed conditions
20.5% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

sonarqubecloud[bot] avatar Apr 22 '24 07:04 sonarqubecloud[bot]

  1. for the same rollup, the indexIds of different partitions are the same, so we must use partition id + index id as shard group id.
  2. compatibility needs to be considered.

wyb avatar May 09 '24 12:05 wyb

[BE Incremental Coverage Report]

:white_check_mark: pass : 9 / 9 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: be/src/storage/lake/transactions.cpp 9 9 100.00% []

github-actions[bot] avatar Jul 08 '24 15:07 github-actions[bot]

Quality Gate Failed Quality Gate failed

Failed conditions
12.7% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

sonarqubecloud[bot] avatar Jul 09 '24 03:07 sonarqubecloud[bot]

[FE Incremental Coverage Report]

:x: fail : 483 / 643 (75.12%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/catalog/PhysicalPartitionImpl.java 0 4 00.00% [143, 144, 145, 146]
:large_blue_circle: com/starrocks/backup/RestoreJob.java 0 1 00.00% [1000]
:large_blue_circle: com/starrocks/alter/AlterJobV2.java 0 3 00.00% [348, 351, 354]
:large_blue_circle: com/starrocks/lake/StarMgrMetaSyncer.java 2 5 40.00% [312, 313, 318]
:large_blue_circle: com/starrocks/catalog/OlapTable.java 2 3 66.67% [1860]
:large_blue_circle: com/starrocks/alter/LakeRollupJob.java 275 408 67.40% [161, 191, 226, 244, 266, 267, 269, 270, 278, 301, 333, 337, 338, 339, 340, 341, 342, 344, 380, 384, 385, 391, 392, 400, 411, 412, 416, 417, 420, 421, 424, 425, 426, 427, 428, 432, 433, 434, 435, 436, 437, 440, 441, 443, 449, 453, 455, 457, 458, 459, 460, 461, 463, 464, 465, 466, 467, 469, 470, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 489, 490, 491, 492, 495, 496, 497, 498, 499, 500, 502, 503, 504, 505, 506, 507, 508, 510, 512, 513, 524, 525, 529, 531, 532, 535, 536, 537, 539, 550, 611, 612, 614, 617, 656, 657, 658, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 699, 701, 702, 703, 704, 705, 708, 709, 710, 711, 712]
:large_blue_circle: com/starrocks/lake/LakeTable.java 3 4 75.00% [184]
:large_blue_circle: com/starrocks/alter/MaterializedViewHandler.java 60 71 84.51% [293, 420, 421, 461, 464, 465, 466, 488, 489, 490, 491]
:large_blue_circle: com/starrocks/alter/LakeTableSchemaChangeJobBase.java 45 46 97.83% [141]
:large_blue_circle: com/starrocks/alter/RollupJobV2.java 65 67 97.01% [524, 561]
:large_blue_circle: com/starrocks/catalog/Partition.java 6 6 100.00% []
:large_blue_circle: com/starrocks/alter/LakeTableAlterJobV2Builder.java 1 1 100.00% []
:large_blue_circle: com/starrocks/catalog/MaterializedIndex.java 4 4 100.00% []
:large_blue_circle: com/starrocks/transaction/LakeTableTxnLogApplier.java 3 3 100.00% []
:large_blue_circle: com/starrocks/sql/analyzer/QueryAnalyzer.java 1 1 100.00% []
:large_blue_circle: com/starrocks/server/LocalMetastore.java 8 8 100.00% []
:large_blue_circle: com/starrocks/alter/LakeTableSchemaChangeJob.java 3 3 100.00% []
:large_blue_circle: com/starrocks/persist/gson/GsonUtils.java 1 1 100.00% []
:large_blue_circle: com/starrocks/alter/AlterJobMgr.java 3 3 100.00% []
:large_blue_circle: com/starrocks/task/AlterReplicaTask.java 1 1 100.00% []

github-actions[bot] avatar Jul 09 '24 05:07 github-actions[bot]