Implement more efficient conversion from Spark column to Comet/Arrow column

[andygrove]

What is the problem the feature request solves?

In CometRowToColumnarExec (soon to be renamed to CometSparkToColumnarExec) we have code for converting from Spark columns to Comet columns, but it it does this by first converting the Spark columns to rows and then converts the rows to Comet columns.

Describe the potential solution

It would be better to implement direct Spark column to Comet column conversion

Additional context

No response

[andygrove]

I am going to take a first pass at this to see what effort is involved.