datafusion-comet
datafusion-comet copied to clipboard
Implement cast between struct types
What is the problem the feature request solves?
Spark supports casting a struct to a different struct type. We should add support in Comet.
private[this] def castStruct(from: StructType, to: StructType): Any => Any = {
val castFuncs: Array[(Any) => Any] = from.fields.zip(to.fields).map {
case (fromField, toField) => cast(fromField.dataType, toField.dataType)
}
// TODO: Could be faster?
buildCast[InternalRow](_, row => {
val newRow = new GenericInternalRow(from.fields.length)
var i = 0
while (i < row.numFields) {
newRow.update(i,
if (row.isNullAt(i)) null else castFuncs(i)(row.get(i, from.apply(i).dataType)))
i += 1
}
newRow
})
}
Describe the potential solution
No response
Additional context
No response
take
We need this feature so that we can support reading structs from Parquet, so I am going to pick this up.
@dharanad Let me know if you are interested in collaborating or reviewing.