datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard
verified publisher icon apache

Improve performance of aggregate expressions by implementing group accumulators

Open andygrove opened this issue 8 months ago • 1 comments

What is the problem the feature request solves?

DataFusion has a fast path for aggregates that implement group accumulators.

See https://github.com/apache/datafusion-comet/issues/1275#issuecomment-2787795808 for one example where we saw a 4x speedup.

We should apply this change to the following Comet aggregates.

  • stddev
  • correlation
  • covariance
  • variance

Describe the potential solution

No response

Additional context

No response

andygrove avatar Apr 08 '25 22:04 andygrove

I would like implement group accumulators variance

dharanad avatar May 15 '25 20:05 dharanad