trino
trino copied to clipboard
Allow for specialized loops only up to 6 parameters
Description
https://github.com/trinodb/trino/pull/19385 (precisely https://github.com/trinodb/trino/commit/046b09decc09a30f6374db58d256b02299d69e84) introduced specialized loops generation for aggregations. The issue with that is the size of the generated method grows exponentially with the number of the operator parameters. It turns out that the org.ow2.asm library used internally by Airlift's bytecode lib has a hardcoded limit for the generated bytecode size (65535 bytes). An aggregation function with 7 parameters results in generated code of 88550 bytes in size and a MethodTooLargeException thrown during instantiation. Apparently there's no built-in aggregation with more than 6 parameters that would allow to catch this issue 🙂
Release notes
( ) This is not user-visible or is docs only, and no release notes are required. (x) Release notes are required. Please propose a release note for me. ( ) Release notes are required, with the following suggested text:
# Section
* Fix some things. ({issue}`issuenumber`)
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla
I've added a test case reproducing the issue, kindly asking for a review @dain 🙂
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.
Thank you for your pull request @a-kramarz
Unfortunately this artificial restriction based on implementation details such as libraries and bytecode generation limits is not the direction we should be going as a project. Our plan instead is to remove the old code and get rid of restrictions. In this case the real solution is to change the generated code to support more arguments. Currently there not much incentive to do that because we only have aggs with 4 arguments.