datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

chore: Preparatory work for implementing custom C2R

Open andygrove opened this issue 6 months ago • 0 comments

Which issue does this PR close?

Closes https://github.com/apache/datafusion-comet/issues/837

Rationale for this change

This is in preparation for implementing an optimized C2R in Comet.

What changes are included in this PR?

  • Add CometColumnarToRowExec which is currently an identical copy of Spark's ColumnarToRowExec but with some comments showing how we could use this to specialize for reading data from CometVector
  • Adds new optimizer rules to replace ColumnarToRowExec with CometColumnarToRowExec
  • Update expected plans

How are these changes tested?

Existing tests

andygrove avatar Aug 16 '24 20:08 andygrove