starrocks
starrocks copied to clipboard
[Feature] Support Skew join in FE
Why I'm doing:
skew values use boradcast join, and other values use shuffle join for skew shuffle join
What I'm doing:
support skew join optimization in FE. This is the first pr of https://github.com/StarRocks/starrocks/pull/48655
Fixes #issue
What type of PR is this:
- [ ] BugFix
- [x] Feature
- [ ] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [ ] Yes, this PR will result in a change in behavior.
- [x] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [x] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [x] 3.3
- [ ] 3.2
- [ ] 3.1
- [ ] 3.0
- [ ] 2.5
[Java-Extensions Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
[FE Incremental Coverage Report]
:white_check_mark: pass : 495 / 560 (88.39%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | com/starrocks/common/util/ProfilingExecPlan.java | 2 | 9 | 22.22% | [244, 245, 246, 247, 248, 249, 251] |
| :large_blue_circle: | com/starrocks/sql/optimizer/operator/physical/PhysicalSplitProduceOperator.java | 6 | 14 | 42.86% | [49, 50, 53, 54, 57, 58, 63, 68] |
| :large_blue_circle: | com/starrocks/sql/optimizer/operator/physical/PhysicalConcatenateOperator.java | 7 | 14 | 50.00% | [41, 53, 54, 57, 58, 61, 62] |
| :large_blue_circle: | com/starrocks/sql/optimizer/operator/physical/PhysicalSplitConsumeOperator.java | 11 | 19 | 57.89% | [65, 66, 69, 70, 73, 74, 79, 84] |
| :large_blue_circle: | com/starrocks/planner/SplitCastPlanFragment.java | 29 | 40 | 72.50% | [63, 85, 86, 91, 92, 96, 97, 101, 102, 106, 107] |
| :large_blue_circle: | com/starrocks/planner/SplitCastDataSink.java | 29 | 34 | 85.29% | [80, 81, 86, 87, 104] |
| :large_blue_circle: | com/starrocks/sql/optimizer/operator/Projection.java | 12 | 14 | 85.71% | [185, 186] |
| :large_blue_circle: | com/starrocks/sql/optimizer/rule/tree/SkewShuffleJoinEliminationRule.java | 193 | 206 | 93.69% | [147, 220, 221, 222, 322, 369, 370, 379, 380, 381, 400, 430, 445] |
| :large_blue_circle: | com/starrocks/qe/scheduler/dag/ExecutionDAG.java | 24 | 25 | 96.00% | [575] |
| :large_blue_circle: | com/starrocks/sql/plan/PlanFragmentBuilder.java | 117 | 120 | 97.50% | [3962, 4078, 4079] |
| :large_blue_circle: | com/starrocks/qe/SessionVariable.java | 8 | 8 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/optimizer/QueryOptimizer.java | 7 | 7 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/optimizer/operator/physical/PhysicalCTEAnchorOperator.java | 4 | 4 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/planner/DataPartition.java | 1 | 1 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/common/LocalExchangerType.java | 8 | 8 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/plan/ExecPlan.java | 2 | 2 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/optimizer/operator/physical/PhysicalHashJoinOperator.java | 4 | 4 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/optimizer/LogicalPlanPrinter.java | 8 | 8 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/analysis/SlotDescriptor.java | 2 | 2 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/optimizer/operator/OperatorType.java | 3 | 3 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/optimizer/rule/implementation/HashJoinImplementationRule.java | 3 | 3 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/planner/UnionNode.java | 8 | 8 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/qe/feedback/guide/JoinTuningGuide.java | 3 | 3 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/common/FeConstants.java | 1 | 1 | 100.00% | [] |
| :large_blue_circle: | com/starrocks/sql/optimizer/OptExpressionVisitor.java | 3 | 3 | 100.00% | [] |
[BE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
