spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-46632][SQL] Fix subexpression elimination when equivalent ternary expressions have different children

Open zml1206 opened this issue 1 year ago • 3 comments

What changes were proposed in this pull request?

Remove unexpected exception thrown in EquivalentExpressions.updateExprInMap(). Equivalent expressions may contain different children, it should happen expression not in map and useCount is -1. For example, before this PR will throw IllegalStateException

Seq((1, 2, 3), (2, 3, 4)).toDF("a", "b", "c")
      .selectExpr("case when a + b + c>3 then 1 when c + a + b>0 then 2 else 0 end as d").show()

Why are the changes needed?

Bug fix.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

New unit test, before this PR will throw IllegalStateException: *** with use count: -1

Was this patch authored or co-authored using generative AI tooling?

No.

zml1206 avatar Apr 19 '24 08:04 zml1206

cc @peter-toth @cloud-fan

zml1206 avatar Apr 19 '24 09:04 zml1206

cc @cloud-fan Do you have time to help take a look? Thank you.

zml1206 avatar Apr 23 '24 01:04 zml1206

@peter-toth Can this PR be merged?

zml1206 avatar May 10 '24 02:05 zml1206

good catch! merging to master/3.5!

cloud-fan avatar Aug 12 '24 08:08 cloud-fan

Thank you. @cloud-fan @peter-toth

zml1206 avatar Aug 12 '24 08:08 zml1206