velox icon indicating copy to clipboard operation
velox copied to clipboard

Published 20 hours ago • verified publisher icon facebookincubator

Add Presto binary lpad and rpad functions

Open soumiiow opened this issue 1 year ago • 8 comments

Adds binary lpad and rpad functions and associated unit tests.

Resolves issue #7328

soumiiow avatar Oct 30 '23 23:10 soumiiow

Deploy Preview for meta-velox canceled.

Name Link
Latest commit 996d6cc1a8dc85730ba804e9719d51f01e7bf095
Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/666cad742429a20008712401

netlify[bot] avatar Oct 30 '23 23:10 netlify[bot]

fuzzer test output:

I20231030 15:50:23.877403 1098637 ExpressionVerifier.cpp:60] At 99 [ null vs null ] I20231030 15:50:23.877418 1098637 ExpressionVerifier.cpp:63] =================== I20231030 15:50:23.877441 1098637 ExpressionVerifier.cpp:74] All results match. I20231030 15:50:23.877470 1098637 ExpressionFuzzer.cpp:1358] ==============================> Done with iteration 4264 I20231030 15:50:23.877579 1098637 ExpressionFuzzer.cpp:1093] ==============================> Top 1 by number of rows processed I20231030 15:50:23.877593 1098637 ExpressionFuzzer.cpp:1095] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20231030 15:50:23.877610 1098637 ExpressionFuzzer.cpp:1099] rpad 43766 100.00% 73556 I20231030 15:50:23.877635 1098637 ExpressionFuzzer.cpp:1105] ==============================> Bottom 1 by number of rows processed I20231030 15:50:23.877653 1098637 ExpressionFuzzer.cpp:1107] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20231030 15:50:23.877668 1098637 ExpressionFuzzer.cpp:1112] rpad 43766 100.00% 73556 I20231030 15:50:23.877683 1098637 ExpressionFuzzer.cpp:1125] ==============================> All stats sorted by number of times the function was chosen I20231030 15:50:23.877701 1098637 ExpressionFuzzer.cpp:1127] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20231030 15:50:23.877717 1098637 ExpressionFuzzer.cpp:1131] rpad 43766 100.00% 73556 [==========] Running 0 tests from 0 test suites. [==========] 0 tests from 0 test suites ran. (0 ms total) [ PASSED ] 0 tests.

I20231030 15:52:50.635804 1101051 ExpressionVerifier.cpp:60] At 99 [ null vs null ] I20231030 15:52:50.635821 1101051 ExpressionVerifier.cpp:63] =================== I20231030 15:52:50.635845 1101051 ExpressionVerifier.cpp:74] All results match. I20231030 15:52:50.635877 1101051 ExpressionFuzzer.cpp:1358] ==============================> Done with iteration 4246 I20231030 15:52:50.636021 1101051 ExpressionFuzzer.cpp:1093] ==============================> Top 1 by number of rows processed I20231030 15:52:50.636035 1101051 ExpressionFuzzer.cpp:1095] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20231030 15:52:50.636052 1101051 ExpressionFuzzer.cpp:1099] lpad 43509 100.00% 73288 I20231030 15:52:50.636072 1101051 ExpressionFuzzer.cpp:1105] ==============================> Bottom 1 by number of rows processed I20231030 15:52:50.636086 1101051 ExpressionFuzzer.cpp:1107] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20231030 15:52:50.636103 1101051 ExpressionFuzzer.cpp:1112] lpad 43509 100.00% 73288 I20231030 15:52:50.636119 1101051 ExpressionFuzzer.cpp:1125] ==============================> All stats sorted by number of times the function was chosen I20231030 15:52:50.636134 1101051 ExpressionFuzzer.cpp:1127] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20231030 15:52:50.636150 1101051 ExpressionFuzzer.cpp:1131] lpad 43509 100.00% 73288 [==========] Running 0 tests from 0 test suites. [==========] 0 tests from 0 test suites ran. (0 ms total) [ PASSED ] 0 tests.

soumiiow avatar Oct 30 '23 23:10 soumiiow

@Yuhta, @kagamiori : Please can you help merge.

aditi-pandit avatar Feb 01 '24 02:02 aditi-pandit

@soumiiow : The linux-build is failing. Please can you debug the failure or rebase your code if needed.

aditi-pandit avatar Mar 15 '24 00:03 aditi-pandit

@soumiiow : The linux-build is failing. Please can you debug the failure or rebase your code if needed.

currently looking through redoing the tests to better test the functions. I'm attempting to cast the strings as varbinary values first so that they do not default to varchars and go into the string implementation of lpad and rpad instead of actually testing the varbinary functions. i pinged you some questions regarding issues im running into offline if you have a quick minute to help with this

soumiiow avatar Mar 15 '24 01:03 soumiiow

@soumiiow : Please can you add a ExpressionFuzzer run with the changes. Would be great to have a more recent run for review.

aditi-pandit avatar May 10 '24 22:05 aditi-pandit

Updated fuzzer test output: I20240510 15:21:23.727039 9623206 FuzzerToolkit.cpp:123] At 99 [ null vs null ] I20240510 15:21:23.727056 9623206 FuzzerToolkit.cpp:126] =================== I20240510 15:21:23.727077 9623206 FuzzerToolkit.cpp:137] Two vectors match. I20240510 15:21:23.727100 9623206 ExpressionFuzzerVerifier.cpp:367] ==============================> Done with iteration 4755 I20240510 15:21:23.727169 9623206 ExpressionFuzzerVerifier.cpp:164] ==============================> Top 1 by number of rows processed I20240510 15:21:23.727185 9623206 ExpressionFuzzerVerifier.cpp:166] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20240510 15:21:23.727200 9623206 ExpressionFuzzerVerifier.cpp:170] lpad 46895 100.00% 87625 I20240510 15:21:23.727221 9623206 ExpressionFuzzerVerifier.cpp:176] ==============================> Bottom 1 by number of rows processed I20240510 15:21:23.727234 9623206 ExpressionFuzzerVerifier.cpp:178] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20240510 15:21:23.727252 9623206 ExpressionFuzzerVerifier.cpp:183] lpad 46895 100.00% 87625 I20240510 15:21:23.727269 9623206 ExpressionFuzzerVerifier.cpp:196] ==============================> All stats sorted by number of times the function was chosen I20240510 15:21:23.727286 9623206 ExpressionFuzzerVerifier.cpp:198] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20240510 15:21:23.727303 9623206 ExpressionFuzzerVerifier.cpp:202] lpad 46895 100.00% 87625 E20240510 15:21:23.727320 9623206 ExpressionFuzzerVerifier.cpp:373] Total iterations: 4756 E20240510 15:21:23.727336 9623206 ExpressionFuzzerVerifier.cpp:374] Total failed: 1276 [==========] Running 0 tests from 0 test suites. [==========] 0 tests from 0 test suites ran. (0 ms total) [ PASSED ] 0 tests.

I20240510 15:23:11.234426 9626599 FuzzerToolkit.cpp:123] At 99 [ null vs null ] I20240510 15:23:11.234439 9626599 FuzzerToolkit.cpp:126] =================== I20240510 15:23:11.234459 9626599 FuzzerToolkit.cpp:137] Two vectors match. I20240510 15:23:11.234480 9626599 ExpressionFuzzerVerifier.cpp:367] ==============================> Done with iteration 4745 I20240510 15:23:11.234548 9626599 ExpressionFuzzerVerifier.cpp:164] ==============================> Top 1 by number of rows processed I20240510 15:23:11.234563 9626599 ExpressionFuzzerVerifier.cpp:166] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20240510 15:23:11.234577 9626599 ExpressionFuzzerVerifier.cpp:170] rpad 46794 100.00% 87406 I20240510 15:23:11.234594 9626599 ExpressionFuzzerVerifier.cpp:176] ==============================> Bottom 1 by number of rows processed I20240510 15:23:11.234609 9626599 ExpressionFuzzerVerifier.cpp:178] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20240510 15:23:11.234622 9626599 ExpressionFuzzerVerifier.cpp:183] rpad 46794 100.00% 87406 I20240510 15:23:11.234637 9626599 ExpressionFuzzerVerifier.cpp:196] ==============================> All stats sorted by number of times the function was chosen I20240510 15:23:11.234652 9626599 ExpressionFuzzerVerifier.cpp:198] Format: functionName numTimesSelected proportionOfTimesSelected numProcessedRows I20240510 15:23:11.234666 9626599 ExpressionFuzzerVerifier.cpp:202] rpad 46794 100.00% 87406 E20240510 15:23:11.234681 9626599 ExpressionFuzzerVerifier.cpp:373] Total iterations: 4746 E20240510 15:23:11.234696 9626599 ExpressionFuzzerVerifier.cpp:374] Total failed: 1273 [==========] Running 0 tests from 0 test suites. [==========] 0 tests from 0 test suites ran. (0 ms total) [ PASSED ] 0 tests.

soumiiow avatar May 10 '24 22:05 soumiiow

@kgpai @kagamiori @Yuhta : Please can you help with review and merge.

aditi-pandit avatar May 10 '24 23:05 aditi-pandit

@soumiiow Can you rebase and fix the failing CI jobs ?

kgpai avatar May 22 '24 20:05 kgpai

@kgpai Please take a look when you can. i rebased the branch and previously fixed the CI build

soumiiow avatar Jun 05 '24 20:06 soumiiow

@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot avatar Jun 05 '24 20:06 facebook-github-bot

Removing the ready-to-merge tag as their are pending changes requested

kevinwilfong avatar Jun 10 '24 17:06 kevinwilfong

@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot avatar Jun 14 '24 21:06 facebook-github-bot

@kevinwilfong merged this pull request in facebookincubator/velox@b5502cfd7e2eeada918f0cf70c8657768383179f.

facebook-github-bot avatar Jun 15 '24 05:06 facebook-github-bot

Conbench analyzed the 1 benchmark run on commit b5502cfd.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

conbench-facebook[bot] avatar Jun 15 '24 06:06 conbench-facebook[bot]