datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

Implement Spark-compatible cast to/from binary type

Open andygrove opened this issue 1 year ago • 2 comments

What is the problem the feature request solves?

https://github.com/apache/datafusion-comet/pull/362 added some tests for casting to/from binary. Some pass and some are ignored because they currently fail.

This issue is for investigating the failures and either adding documentation to explain why we are incompatible, or making them compatible.

Describe the potential solution

No response

Additional context

No response

andygrove avatar May 03 '24 13:05 andygrove

I would like to work on this

mattharder91 avatar May 04 '24 08:05 mattharder91

Thanks @mattharder91. Feel free to break this down into smaller issues if needed e.g. string <-> binary, integers <-> binary and so on.

There is a little more info on binary -> string in https://github.com/apache/datafusion-comet/pull/379. It looks this works for valid UTF-8 inputs but Spark allows non UTF-8 input.

andygrove avatar May 04 '24 14:05 andygrove