trino icon indicating copy to clipboard operation
trino copied to clipboard

Improve determinism by preserving split order

Open gaurav8297 opened this issue 1 year ago • 4 comments
trafficstars

Description

In this we improve query execution determinism by
preserving split order when scheduling
in pipeline execution mode.

This is essentially needed such that splits coming out of CacheSplitSource preserve the order
between coordinator and workers. This way we
prevent scheduling two splits with same
CacheSplitId to reuse cache within a query.

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required. ( ) Release notes are required. Please propose a release note for me. ( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

gaurav8297 avatar Jun 21 '24 17:06 gaurav8297

Started benchmark workflow for this PR with test type = iceberg/sf10000_parquet_part.

Building Trino finished with status: success Benchmark finished with status: failure Status message:

Started benchmark workflow for this PR with test type = hive/sf1000_parquet_part.

Building Trino finished with status: success Benchmark finished with status: success Status message: No baseline found. Benchmark Comparison Report

CI issue: https://github.com/trinodb/trino/issues/18697#issue-1853098596

gaurav8297 avatar Jun 26 '24 05:06 gaurav8297

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_part.

Building Trino finished with status: success Benchmark finished with status: success Status message: NO Regression found. Benchmark Comparison Report