datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

Accelerate TPC-H without any fallbacks to Spark

Open andygrove opened this issue 3 months ago • 0 comments

What is the problem the feature request solves?

Comet now accelerates most of TPC-H, but there are still a small number of fallbacks happening.

q11/20:

WARN CometSparkSessionExtensions: Comet cannot accelerate Cast because: cast(ps_availqty#343 as decimal(10,0)) is not fully compatible with Spark (No overflow check)

q11:

WARN CometSparkSessionExtensions: Comet cannot accelerate LocalTableScanExec because: LocalTableScan is not supported

q16:

WARN CometSparkSessionExtensions: Comet cannot accelerate BroadcastHashJoinExec because: BuildRight with LeftAnti is not supported

q21:

WARN CometSparkSessionExtensions: Comet cannot accelerate SortMergeJoinExec because: spark.comet.exec.sortMergeJoinWithJoinFilter.enabled is not enabled

Issues:

  • [ ] https://github.com/apache/datafusion-comet/issues/1909
  • [x] https://github.com/apache/datafusion-comet/issues/2049
  • [ ] https://github.com/apache/datafusion-comet/issues/901
  • [x] #2513

Describe the potential solution

No response

Additional context

No response

andygrove avatar Sep 26 '25 15:09 andygrove