trino
trino copied to clipboard
Published
20 hours ago •
trinodb
trinodb
[WIP] Fault tolerant scheduler 2.0
Description
TODO
Non-technical explanation
TODO
Release notes
(x) This is not user-visible or docs only and no release notes are required. ( ) Release notes are required, please propose a release note for me. ( ) Release notes are required, with the following suggested text:
# Section
* Fix some things. ({issue}`issuenumber`)
Benchmark results:
+----------------------+-----------------------+----------------------+-----------------------+----------+-----------+
| base_cpu_time_millis | base_wall_time_millis | test_cpu_time_millis | test_wall_time_millis | cpu_diff | wall_diff |
+----------------------+-----------------------+----------------------+-----------------------+----------+-----------+
| 18184316267 | 77292978 | 18536131298 | 76952203 | 1.01935 | 0.99559 |
+----------------------+-----------------------+----------------------+-----------------------+----------+-----------+
+-------------------------------+----------------------+-----------------------+----------------------+-----------------------+----------+-----------+
| suite | base_cpu_time_millis | base_wall_time_millis | test_cpu_time_millis | test_wall_time_millis | cpu_diff | wall_diff |
+-------------------------------+----------------------+-----------------------+----------------------+-----------------------+----------+-----------+
| tpcds_sf10000_partitioned | 2151027639 | 10666623 | 2192108379 | 10325315 | 1.01910 | 0.96800 |
| tpcds_sf10000_partitioned_etl | 12575752138 | 45907876 | 12749500471 | 45367419 | 1.01382 | 0.98823 |
| tpcds_sf100_partitioned | 25285611 | 1298684 | 24642584 | 1285296 | 0.97457 | 0.98969 |
| tpcds_sf100_partitioned_etl | 112666635 | 5309658 | 116431690 | 5588347 | 1.03342 | 1.05249 |
| tpch_sf10000_bucketed | 833789592 | 3341612 | 860208652 | 3195566 | 1.03169 | 0.95629 |
| tpch_sf10000_bucketed_etl | 2459310174 | 9295363 | 2565149397 | 9585205 | 1.04304 | 1.03118 |
| tpch_sf100_bucketed | 6506475 | 168296 | 5905512 | 160160 | 0.90764 | 0.95166 |
| tpch_sf100_bucketed_etl | 19978003 | 1304866 | 22184613 | 1444895 | 1.11045 | 1.10731 |
+-------------------------------+----------------------+-----------------------+----------------------+-----------------------+----------+-----------+
Detailed: https://gist.github.com/arhimondr/02ccc06a4f145fcd5f47b91ff068c013
Still on top of several PRs (#14328, #14329, #14330, #14320) but is ready for review.
Thanks for the review. Went through the first section of comments. Going to continue tomorrow.