apache
IGNITE-14777 window functions support
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:
The Contribution Checklist
- [X] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the Patch Available state.
- [X] The pull request body describes changes that have been made. The description explains WHAT and WHY was made instead of HOW.
- [X] The pull request title is treated as the final commit message.
The following pattern must be used:
IGNITE-XXXX Change summarywhereXXXX- number of JIRA issue. - [ ] A reviewer has been mentioned through the JIRA comments (see the Maintainers list)
- [ ] The pull request has been checked by the Teamcity Bot and
the
green visaattached to the JIRA ticket (see TC.Bot: Check PR)
Notes
If you need any help, please email [email protected] or ask anу advice on http://asf.slack.com #ignite channel.
Description of Changes:
Added support for planning and executing window functions. Added special window functions: row_number, rank, dense_rank, percent_rank, cume_dist, ntile, nth_value, first_value, last_value, lag, lead. Provides two modes of window function execution:
- With partition buffering
- Without buffering, accessing only the current and previous row (only for row_number, rank, dense_rank, and standard aggregates with the window specification ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)
Supports specifying integer offsets in either variant (ROWS / RANGE) and time interval offsets for RANGE.
Window Planning:
During query planning, windows are split into separate rels for each group of aggregation functions. Each logical window rel includes a collation that is required for correctly partitioning rows and defining frames when computing the window. Splitting is done using Calcite’s standard rule, which groups function calls based on the window specification (according to the OVER clause). After that, constants used in the window are projected to support referencing them in the current implementation of Ignite aggregates. (If constants are used in FOLLOWING/PRECEDING, they are directly substituted into the offset, which helps reduce the number of frame boundary searches.) An additional planning phase was introduced specifically for window planning. (I couldn't find a suitable existing place for the new rules, so I followed the approach used in Apache Drill.)
Separate Change:
During development, when attempting to upgrade to Calcite 1.39, it was discovered that IgniteTypeFactory#leastRestrictive does not take into account the nullability of the resulting type when merging FLOAT and DOUBLE.
P.S. I'll be appreciate to any feedback
@oleg-zinovev, I've partially reviewed your PR. Review not completed yet, but I have some comments to publish. Also there are a lot of codestyle violations. Please read the https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines article about Ignite codestyle. Most of the problems can be detected automatically (for example using command: mvn clean install -DskipTests -Dmaven.javadoc.skip=true -Pcheckstyle -T 1C), but there are also some violated rules that currently not checked by maven, such as:
- Comments should be started with an upper-case letter and ended with point
- Braces should not be used for
ifone-line statements - No empty lines allowed after class declaration
- Comments like
/** */(two spaces inside) are not allowed, use one space - Abbreviation should be used for field/variables like 'value', 'group', 'buffer', ..., see full list here: https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules
@alex-plekhanov, thanks for the review. I’ll try to fix the issues or follow up with some questions during the week, by August 4th.
@alex-plekhanov
I have a few questions regarding the ProjectWindowConstantsRule and the additional query planning phase (I added them in the comments). If possible, could you please answer them before I start working on the remarks about the WindowNode implementation and the trait derivation/passthrough logic?