paimon
paimon copied to clipboard
[core] Skip processed sequence group fields to improve performance of PartialUpdateMergeFunction
Purpose
Skip processed sequence group fields to improve performance of PartialUpdateMergeFunction
- skip null sequence group fields
- skip sequence group fields have been updated at once
- move
seqComparator != nullcheck forward for common case optimization
Tests
- PartialUpdateMergeFunctionTest
- PartialUpdateMergeFunctionBenchmark new
Running benchmark: partial-update-benchmark
Running case: updateWithSequenceGroup
Iteration 0 took 13835399 microseconds
Iteration 1 took 13355838 microseconds
Iteration 2 took 13048940 microseconds
Iteration 3 took 13016992 microseconds
Iteration 4 took 13141102 microseconds
Stopped after 5 iterations, 66398 ms
OpenJDK 64-Bit Server VM 1.8.0_412-b08 on Mac OS X 14.4.1
Apple M1 Pro
partial-update-benchmark: Best/Avg Time(ms) Row Rate(K/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATORTEST_partial-update-benchmark_updateWithSequenceGroup 13017 / 13280 3072.9 325.4 1.0X
Running benchmark: partial-update-benchmark
Running case: retractWithSequenceGroup
Iteration 0 took 12762749 microseconds
Iteration 1 took 12572677 microseconds
Iteration 2 took 12441134 microseconds
Iteration 3 took 12490136 microseconds
Iteration 4 took 12677572 microseconds
Stopped after 5 iterations, 62944 ms
OpenJDK 64-Bit Server VM 1.8.0_412-b08 on Mac OS X 14.4.1
Apple M1 Pro
partial-update-benchmark: Best/Avg Time(ms) Row Rate(K/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATORTEST_partial-update-benchmark_retractWithSequenceGroup 12441 / 12589 3215.1 311.0 1.0X
Running benchmark: partial-update-benchmark
Running case: updateWithEmptySequenceGroup
Iteration 0 took 10842099 microseconds
Iteration 1 took 10364416 microseconds
Iteration 2 took 10588347 microseconds
Iteration 3 took 10492397 microseconds
Iteration 4 took 10618167 microseconds
Stopped after 5 iterations, 52905 ms
OpenJDK 64-Bit Server VM 1.8.0_412-b08 on Mac OS X 14.4.1
Apple M1 Pro
partial-update-benchmark: Best/Avg Time(ms) Row Rate(K/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATORTEST_partial-update-benchmark_updateWithEmptySequenceGroup 10364 / 10581 3859.4 259.1 1.0X
old
Running benchmark: partial-update-benchmark
Running case: updateWithSequenceGroup
Iteration 0 took 14191378 microseconds
Iteration 1 took 14021648 microseconds
Iteration 2 took 14039543 microseconds
Iteration 3 took 13882215 microseconds
Iteration 4 took 13819146 microseconds
Stopped after 5 iterations, 69953 ms
OpenJDK 64-Bit Server VM 1.8.0_412-b08 on Mac OS X 14.4.1
Apple M1 Pro
partial-update-benchmark: Best/Avg Time(ms) Row Rate(K/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATORTEST_partial-update-benchmark_updateWithSequenceGroup 13819 / 13991 2894.5 345.5 1.0X
Running benchmark: partial-update-benchmark
Running case: retractWithSequenceGroup
Iteration 0 took 14462734 microseconds
Iteration 1 took 14674714 microseconds
Iteration 2 took 15522533 microseconds
Iteration 3 took 15111800 microseconds
Iteration 4 took 14651207 microseconds
Stopped after 5 iterations, 74422 ms
OpenJDK 64-Bit Server VM 1.8.0_412-b08 on Mac OS X 14.4.1
Apple M1 Pro
partial-update-benchmark: Best/Avg Time(ms) Row Rate(K/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATORTEST_partial-update-benchmark_retractWithSequenceGroup 14463 / 14885 2765.7 361.6 1.0X
Running benchmark: partial-update-benchmark
Running case: updateWithEmptySequenceGroup
Iteration 0 took 10743736 microseconds
Iteration 1 took 11192192 microseconds
Iteration 2 took 12587153 microseconds
Iteration 3 took 11009585 microseconds
Iteration 4 took 10661533 microseconds
Stopped after 5 iterations, 56194 ms
OpenJDK 64-Bit Server VM 1.8.0_412-b08 on Mac OS X 14.4.1
Apple M1 Pro
partial-update-benchmark: Best/Avg Time(ms) Row Rate(K/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
OPERATORTEST_partial-update-benchmark_updateWithEmptySequenceGroup 10662 / 11239 3751.8 266.5 1.0X
@JingsongLi All CI succeeded extra non-related hive metastore issue, PTAL, thanks :)
CI succeeded in my repo.
@JingsongLi CI succeeded
@yuzelin Hi, PTAL, thanks:)