trino icon indicating copy to clipboard operation
trino copied to clipboard

Order split scheduling between stages for common subqueries

Open sopel39 opened this issue 9 months ago • 0 comments

Subquery s1 and s2 might have common subquery extracted. However, s1 and s2 might have limited number of common splits. Therefore it doesn’t make sense for stage s1 to wait for stage s2 to be completed (and vice-versa). It makes more sense to have some kind of smart logic to order execution of common splits so that split with id is not processed at the same time by s1 and s2. This could maybe be achieved by improving CacheSplitSource so that it’s aware of split sources s1 and s2 and orders split execution in smart way.

sopel39 avatar May 24 '24 14:05 sopel39