datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

Remove COMET_SHUFFLE_FALLBACK_TO_COLUMNAR config

Open andygrove opened this issue 11 months ago • 0 comments

What is the problem the feature request solves?

https://github.com/apache/datafusion-comet/pull/1209 enabled accelerating more queries with Comet but also exposed some new bugs so a COMET_SHUFFLE_FALLBACK_TO_COLUMNAR config was added as a temporary workaround to avoid Spark SQL tests failing.

We should remove this config once the bugs are fixed so that we test more of the Spark SQL queries with Comet shuffle.

Describe the potential solution

No response

Additional context

No response

andygrove avatar Jan 09 '25 16:01 andygrove