blaze icon indicating copy to clipboard operation
blaze copied to clipboard

`spark.blaze.enable.shuffle` doesn't work

Open PengleiShi opened this issue 2 years ago • 2 comments

Describe the bug I try to disable coverting ShuffleExchangeExec, but it doesn't work. And i found the code

def convertSparkPlan(exec: SparkPlan): SparkPlan = {
    exec match {
      case e: ShuffleExchangeExec => tryConvert(e, convertShuffleExchangeExec)
    .....

It's not dependent on the configuration spark.blaze.enable.shuffle. To Reproduce Add configuration --conf spark.blaze.enable.shuffle=false when submit

Expected behavior Blaze extension should not covert ShuffleExchangeExec if spark.blaze.enable.shuffle is true.

Additional context

PengleiShi avatar Jul 21 '22 12:07 PengleiShi

ping @richox

PengleiShi avatar Jul 21 '22 12:07 PengleiShi

ping @yjshen

PengleiShi avatar Jul 28 '22 08:07 PengleiShi