deephaven-core icon indicating copy to clipboard operation
deephaven-core copied to clipboard

Improve `AggFormula` performance by re-using `GroupByChunkedOperator` rather than re-creating new operators.

Open lbooker42 opened this issue 1 year ago • 0 comments

Initial provision is made to re-use a shared GroupByChunkedOperator in FormulaMultiColumnChunkedOperator and FormulaChunkedOperator but more work is needed before this can be fully used.

  1. We need to optimize the operator order in AggregationOptimizer to look for GroupByChunkedOperator instances from group aggs or that may be created for other formulas.
  2. For FormulaMultiColumnChunkedOperator, we need to update GroupByChunkedOperator to keep two sets of output columns: public columns for requested GroupBy aggs and "hidden" ones only exposed to internal operators.

lbooker42 avatar Nov 11 '24 20:11 lbooker42