datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

docs: Document incompatibility of unhex with Spark 3.2

Open andygrove opened this issue 1 year ago • 0 comments

What is the problem the feature request solves?

https://github.com/apache/datafusion-comet/pull/342 adds support for unhex, and we see compatible behavior in Comet with Spark 3.3 and above.

However, when running against Spark 3.2, we include a bug fix for https://issues.apache.org/jira/browse/SPARK-40924 that was added in Spark 3.3, so although Comet's behavior is more correct when running against Spark 3.2, it is not the same, and we should document this.

Rather than manually add a note in the compatibility guide I would prefer to extend the mechanism we are using for documenting cast expressions so that we can automate generating compatibility notes for other expression types.

Describe the potential solution

No response

Additional context

No response

andygrove avatar May 08 '24 14:05 andygrove