starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Enhancement] Use shuffle bucket join to reduce data shuffle

Open Youngwb opened this issue 3 years ago • 4 comments

What type of PR is this:

  • [ ] bug
  • [ ] feature
  • [x] enhancement
  • [ ] refactor
  • [ ] others

Which issues of this PR fixes :

Fixes #9790

Problem Summary(Required) :

Main change:

  1. Satisty requirement for SHFFLE_JOIN property, hash by column A can satisfy hash by A,B columns
  2. check plan is legal for shuffle join in ChildOutputPropertyGuarantor. if not, add or modify the enforcer to be legal

Checklist:

  • [ ] I have added test cases for my bug fix or my new feature
  • [ ] I have added user document for my new feature or new function

Youngwb avatar Aug 09 '22 11:08 Youngwb

run starrocks_fe_unittest

Youngwb avatar Aug 10 '22 02:08 Youngwb

[FE PR Coverage Check]

:heart_eyes: pass : 50 / 55 (90.91%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/sql/optimizer/ChildOutputPropertyGuarantor.java 49 54 90.74% [146, 147, 148, 149, 150]
:large_blue_circle: com/starrocks/sql/optimizer/base/HashDistributionDesc.java 1 1 100.00% []

wanpengfei-git avatar Aug 10 '22 02:08 wanpengfei-git

run starrocks_admit_test

Youngwb avatar Aug 10 '22 06:08 Youngwb

run starrocks_admit_test

wanpengfei-git avatar Aug 12 '22 12:08 wanpengfei-git