gazelle_plugin
gazelle_plugin copied to clipboard
Core Dump(Code:134) with WSCG SetDependencies function fail when running TPC-DS q84
Describe the bug Core Dump(Code:134) with a TypedWholeStageCodeGenImpl::WholeStageCodeGenResultIterator::SetDependencies function fail when running TPC-DS q84
To Reproduce This issue can be reproduced in below steps: Under SF500 testing in a single server,
- There is a four continue BHJ in q84 if set autoBroadcastJoinThreshold to "60M"
- We need to set spark.sql.autoBroadcastJoinThreshold to a certain value in my case is "1000k", let 4 BHJs become SMJ -> BHJ -> BHJ -> SMJ and trigger this issue.
- If set the value too small, all BHJ will be replaced to SMJ and no issue in this case(4 SMJs).
Expected behavior
If we don't support BHJ -> SMJ scenario, then we can set up a rule to eliminate this case.
Additional context Add any other context about the problem here.