pinot icon indicating copy to clipboard operation
pinot copied to clipboard

Added GENERATE_ARRAY function

Open aadilkhalifa opened this issue 1 year ago • 6 comments

Pull request for the issue: https://github.com/apache/pinot/issues/11786

Adding ARRAY_GENERATE function to allow generation of array of values with start, end and step value (optional). Supported data types are int, long, float and double. Currently does not support multi-stage query engine.

Eg. SELECT GENERATE_ARRAY(1, 5, 1) from table limit 1;

aadilkhalifa avatar Jan 21 '24 10:01 aadilkhalifa

Thanks @aadilkhalifa for raising the PR!

Can you add some integration tests under org.apache.pinot.integration.tests.custom.ArrayTest

xiangfu0 avatar Jan 21 '24 19:01 xiangfu0

Codecov Report

Attention: Patch coverage is 41.93548% with 126 lines in your changes missing coverage. Please review.

Project coverage is 62.09%. Comparing base (59551e4) to head (cf17495). Report is 738 commits behind head on master.

Files Patch % Lines
...sform/function/GenerateArrayTransformFunction.java 46.90% 84 Missing and 19 partials :warning:
...e/pinot/common/function/scalar/ArrayFunctions.java 0.00% 20 Missing :warning:
...r/transform/function/TransformFunctionFactory.java 0.00% 2 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #12293      +/-   ##
============================================
+ Coverage     61.75%   62.09%   +0.34%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2559     +123     
  Lines        133233   141154    +7921     
  Branches      20636    21915    +1279     
============================================
+ Hits          82274    87646    +5372     
- Misses        44911    46873    +1962     
- Partials       6048     6635     +587     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration <0.01% <0.00%> (-0.01%) :arrow_down:
integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration2 0.00% <0.00%> (ø)
java-11 62.05% <41.93%> (+0.34%) :arrow_up:
java-21 61.96% <41.93%> (+0.33%) :arrow_up:
skip-bytebuffers-false 62.07% <41.93%> (+0.32%) :arrow_up:
skip-bytebuffers-true 61.94% <41.93%> (+34.21%) :arrow_up:
temurin 62.09% <41.93%> (+0.34%) :arrow_up:
unittests 62.08% <41.93%> (+0.34%) :arrow_up:
unittests1 46.69% <41.93%> (-0.20%) :arrow_down:
unittests2 27.62% <0.00%> (-0.11%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Jan 21 '24 19:01 codecov-commenter

Thanks @aadilkhalifa for raising the PR!

Can you add some integration tests under org.apache.pinot.integration.tests.custom.ArrayTest

@xiangfu0 I've added the integration tests.

aadilkhalifa avatar Jan 27 '24 07:01 aadilkhalifa

Shall we keep the same name as BQ: generateArray. For the scalar function version, we can do generateIntArray, generateLongArray etc.

+1, @aadilkhalifa can you update the function names?

xiangfu0 avatar Mar 14 '24 18:03 xiangfu0

@Jackie-Jiang @xiangfu0 I've updated the names.

aadilkhalifa avatar Mar 21 '24 08:03 aadilkhalifa

can this be merged and issue closed?

npawar avatar May 23 '24 00:05 npawar

can this be merged and issue closed?

@xiangfu0 Can this be merged?

aadilkhalifa avatar Jul 10 '24 16:07 aadilkhalifa