trino icon indicating copy to clipboard operation
trino copied to clipboard

Fix table function execution without partitioning

Open findepi opened this issue 2 years ago • 5 comments

Previously, when table function did not declare partitioning, it would be globally distributed, but on a worker node it would run single-threaded and first buffer all data in memory, like a one big WINDOW. After the change, the local execution processes input pages in a streaming fashion.

This commit also fixes property derivations for a case where table function is partitioned on empty list of symbols (global grouping).

Fixes https://github.com/trinodb/trino/issues/20398

findepi avatar Apr 03 '24 19:04 findepi

cc @ebyhr @hovaesco

findepi avatar Apr 03 '24 19:04 findepi

just rebased, no other changes

findepi avatar Apr 05 '24 10:04 findepi

@martint @ebyhr PTAL

findepi avatar Apr 05 '24 10:04 findepi

( rebased, no other changes )

findepi avatar Apr 15 '24 14:04 findepi

draft -- this implementation has suboptimal lifecycle for TableFunctionDataProcessor https://github.com/trinodb/trino/pull/21558 attempts to improve this

findepi avatar Apr 15 '24 15:04 findepi

Any traction on this? We have been running a fork with these changes and would love for it to be in trino proper.

tbaeg avatar Dec 30 '24 15:12 tbaeg

@kasiafi @findepi .. did you want to continue on this or close it?

mosabua avatar Jan 28 '25 21:01 mosabua

@mosabua I think we can close - I would be opening a new PR for this.

Praveen2112 avatar Feb 05 '25 09:02 Praveen2112

This pull request has gone a while without any activity. Tagging for triage help: @mosabua

github-actions[bot] avatar Feb 26 '25 17:02 github-actions[bot]