trino icon indicating copy to clipboard operation
trino copied to clipboard

Improve stats reporting for group by operator

Open sopel39 opened this issue 9 months ago • 2 comments

Similarly as in https://github.com/trinodb/trino/commit/f46fd9c1dccb975bf480f8c7029daf7e45541b54 we could separately report hash lookups/updates and aggregations' accumulator updates to get more insight into query bottlenecks.

cc @dain @raunaqmorarka

sopel39 avatar May 10 '24 15:05 sopel39

I want to take this up, @sopel39 can you give more insight of this.

sug-ghosh avatar May 22 '24 09:05 sug-ghosh

I want to take this up, @sopel39 can you give more insight of this.

Sure go ahead. Take a look at commit https://github.com/trinodb/trino/commit/f46fd9c1dccb975bf480f8c7029daf7e45541b54. You can see that ScanFilterAndProjectOperator keeps projection and fitltering stats in PageProcessorMetrics, which are then returned as operator metrics. Similar approach could be applied for group by operator. We can measure performance of GroupByHash and Accumulator separately.

sopel39 avatar May 24 '24 10:05 sopel39

okay.

sug-ghosh avatar Jun 04 '24 06:06 sug-ghosh

@sug-ghosh are you working on this or planning to? If not, I'd be happy to take it on.

piotrrzysko avatar Oct 20 '24 08:10 piotrrzysko