druid icon indicating copy to clipboard operation
druid copied to clipboard

Log the number of bytes returned from historical when max scatter-gather is reached

Open TessaIO opened this issue 2 weeks ago • 1 comments

Description

Currently, when the query returned data is bigger than the max scatter-gather, we just log that the query's max scatter-gather bytes limit is reached, but we don't have visibility on how big is the data returned from the historical which makes Druid administration hard.

Motivation

We have some queries that are different in their nature and a max scatter-gather value can fit one query but not all. We want to this value on the context of the query so it's pretty customized. And for that, we need to know how much each query needed when it fails so we can adjust this value.

TessaIO avatar Dec 12 '25 17:12 TessaIO

You can assign it to me as I will work on it

TessaIO avatar Dec 12 '25 17:12 TessaIO