Support forcing pre-partitioned exchanges for window functions to avoid data shuffle using query hint

[yashmayya]

• Adds a new window option query hint is_partitioned_by_window_keys that can be used to force pre-partitioned exchanges for window functions in order to avoid data shuffle when we know that the input is already partitioned by the window's partition keys.
• In most scenarios where the input is a table that is explicitly partitioned by the window's partition keys, we can use the /*+ tableOptions(partition_key='...', partition_size='...') */ hint on the table scan so that the planner can automatically determine that the partitioning matches and the two stages should be colocated. However, in certain scenarios like implicit partitioning and also for debugging / iterating on queries, this new hint to force enable or disable the pre-partitioned exchanges can be useful.
• Note that no tests are added here, because window hints are currently broken in Calcite (https://issues.apache.org/jira/browse/CALCITE-7338). So the functionality introduced in this PR depends on a fix for the linked Calcite issue.

[codecov-commenter]

Codecov Report

:x: Patch coverage is 75.00000% with 4 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 63.26%. Comparing base (42715fc) to head (059cf4d). :warning: Report is 12 commits behind head on master.

Files with missing lines	Patch %	Lines
...lcite/rel/rules/PinotRelDistributionTraitRule.java	60.00%	0 Missing and 2 partials :warning:
...pache/pinot/calcite/rel/hint/PinotHintOptions.java	66.66%	0 Missing and 1 partial :warning:
.../calcite/rel/logical/PinotLogicalSortExchange.java	75.00%	1 Missing :warning:

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #17395      +/-   ##
============================================
+ Coverage     63.21%   63.26%   +0.05%     
  Complexity     1474     1474              
============================================
  Files          3147     3147              
  Lines        187562   187590      +28     
  Branches      28712    28716       +4     
============================================
+ Hits         118560   118672     +112     
+ Misses        59808    59733      -75     
+ Partials       9194     9185       -9     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-11	63.20% <75.00%> (+0.01%)	:arrow_up:
java-21	63.20% <75.00%> (+0.02%)	:arrow_up:
temurin	63.26% <75.00%> (+0.05%)	:arrow_up:
unittests	63.25% <75.00%> (+0.05%)	:arrow_up:
unittests1	55.66% <75.00%> (+0.03%)	:arrow_up:
unittests2	33.90% <6.25%> (+0.02%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[ankitsultana]

@yashmayya : have you folks tried the physical optimizer? It can colocate whenever applicable. It's missing some features though like dynamic filter for semijoin, etc. though.

[yashmayya]

@ankitsultana we haven't fully evaluated the physical optimizer internally just yet, it is on our radar though.

It can colocate whenever applicable

Even for window functions where the input table is not explicitly partitioned by the window's partition keys?