datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard
verified publisher icon apache

org.apache.spark.sql.catalyst.expressions.BoundReference cannot be cast to class org.apache.spark.sql.ColumnarExpression

Open kazuyukitanimura opened this issue 1 year ago • 2 comments

Describe the bug

In SparkSessionExtensionSuite, there are 4 test failures with org.apache.spark.sql.catalyst.expressions.BoundReference cannot be cast to class org.apache.spark.sql.ColumnarExpression when spark.comet.enabled=true is set.

inject columnar AQE on inject columnar AQE off SPARK-39991: AQE should retain column statistics from completed query stages reset column vectors

Steps to reproduce

SparkSessionExtensionSuite is not using SharedSparkSessionBase, so add spark.comet.enabled=true by changing code

Expected behavior

No response

Additional context

No response

kazuyukitanimura avatar Dec 24 '24 11:12 kazuyukitanimura

@kazuyukitanimura Is this issue still valid?

andygrove avatar Apr 17 '25 20:04 andygrove

@andygrove Yes, I think so https://github.com/apache/datafusion-comet/blob/e7a3214510091cf9d177fabdf9e3221317d1e785/dev/diffs/3.4.3.diff#L981

kazuyukitanimura avatar Apr 18 '25 18:04 kazuyukitanimura