datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

Implement cast between struct types

Open andygrove opened this issue 1 year ago • 1 comments

What is the problem the feature request solves?

Spark supports casting a struct to a different struct type. We should add support in Comet.

  private[this] def castStruct(from: StructType, to: StructType): Any => Any = {
    val castFuncs: Array[(Any) => Any] = from.fields.zip(to.fields).map {
      case (fromField, toField) => cast(fromField.dataType, toField.dataType)
    }
    // TODO: Could be faster?
    buildCast[InternalRow](_, row => {
      val newRow = new GenericInternalRow(from.fields.length)
      var i = 0
      while (i < row.numFields) {
        newRow.update(i,
          if (row.isNullAt(i)) null else castFuncs(i)(row.get(i, from.apply(i).dataType)))
        i += 1
      }
      newRow
    })
  }

Describe the potential solution

No response

Additional context

No response

andygrove avatar Aug 12 '24 17:08 andygrove

take

dharanad avatar Aug 13 '24 18:08 dharanad

We need this feature so that we can support reading structs from Parquet, so I am going to pick this up.

@dharanad Let me know if you are interested in collaborating or reviewing.

andygrove avatar Nov 11 '24 14:11 andygrove