datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

Should ScanExec use Spark-compatible cast instead of DataFusion cast?

Open andygrove opened this issue 1 year ago • 0 comments

What is the problem the feature request solves?

In ScanExec there can be a difference between the Parquet data type and the Spark data type being requested. In this case, we currently delegate to DataFusion's cast_with_options, but should we be using Comet's spark_cast method instead?

Describe the potential solution

No response

Additional context

No response

andygrove avatar Aug 10 '24 15:08 andygrove