gazelle_plugin icon indicating copy to clipboard operation
gazelle_plugin copied to clipboard

Core Dump(Code:134) with WSCG SetDependencies function fail when running TPC-DS q84

Open weiting-chen opened this issue 2 years ago • 0 comments

Describe the bug Core Dump(Code:134) with a TypedWholeStageCodeGenImpl::WholeStageCodeGenResultIterator::SetDependencies function fail when running TPC-DS q84

To Reproduce This issue can be reproduced in below steps: Under SF500 testing in a single server,

  1. There is a four continue BHJ in q84 if set autoBroadcastJoinThreshold to "60M"
  2. We need to set spark.sql.autoBroadcastJoinThreshold to a certain value in my case is "1000k", let 4 BHJs become SMJ -> BHJ -> BHJ -> SMJ and trigger this issue.
  3. If set the value too small, all BHJ will be replaced to SMJ and no issue in this case(4 SMJs).

Expected behavior If we don't support BHJ -> SMJ scenario, then we can set up a rule to eliminate this case. 20221102_SelfTest_DAG

Additional context Add any other context about the problem here.

image

weiting-chen avatar Nov 02 '22 07:11 weiting-chen