velox icon indicating copy to clipboard operation
velox copied to clipboard

Improve normalized key group probe

Open wypb opened this issue 1 year ago • 1 comments

Improve normalized key group probe by increase group size from 4 to 64 in groupProbe and groupNormalizedKeyProbe. TPCDS sf100000 (100TB) Q95 can be improved by about 6% through this optimization.

BEFORE:

=================================================================================
[...]benchmarks/HashJoinGroupProbeBenchmark.cpp     relative  time/iter   iters/s
=================================================================================
ARRAY,Size:100000,withDup:false                                  6.48ms    154.43
ARRAY,Size:100000,withDup:true                                   1.76ms    567.06
ARRAY,Size:1048573,withDup:false                                51.28ms     19.50
ARRAY,Size:1048573,withDup:true                                 17.16ms     58.26
NORMALIZED_KEY,Size:100000,withDup:false                        11.77ms     84.97
NORMALIZED_KEY,Size:100000,withDup:true                          6.92ms    144.52
NORMALIZED_KEY,Size:2097149,withDup:false                      192.61ms      5.19
NORMALIZED_KEY,Size:2097149,withDup:true                       194.08ms      5.15
NORMALIZED_KEY,Size:16777216,withDup:false                        2.09s   479.04m
NORMALIZED_KEY,Size:16777216,withDup:true                         1.58s   630.95m
HASH,Size:2097149,withDup:false                                230.50ms      4.34
HASH,Size:2097149,withDup:true                                 234.45ms      4.27
HASH,Size:16777216,withDup:false                                  2.07s   483.03m
HASH,Size:16777216,withDup:true                                   1.97s   506.43m

AFTER:

=================================================================================
[...]benchmarks/HashJoinGroupProbeBenchmark.cpp     relative  time/iter   iters/s
=================================================================================
ARRAY,Size:100000,withDup:false                                  6.85ms    145.98
ARRAY,Size:100000,withDup:true                                   1.77ms    565.84
ARRAY,Size:1048573,withDup:false                                54.08ms     18.49
ARRAY,Size:1048573,withDup:true                                 17.67ms     56.59
NORMALIZED_KEY,Size:100000,withDup:false                        11.49ms     87.06
NORMALIZED_KEY,Size:100000,withDup:true                          6.76ms    147.87
NORMALIZED_KEY,Size:2097149,withDup:false                      165.09ms      6.06
NORMALIZED_KEY,Size:2097149,withDup:true                       166.43ms      6.01
NORMALIZED_KEY,Size:16777216,withDup:false                        1.40s   714.74m
NORMALIZED_KEY,Size:16777216,withDup:true                         1.43s   701.20m
HASH,Size:2097149,withDup:false                                204.48ms      4.89
HASH,Size:2097149,withDup:true                                 203.01ms      4.93
HASH,Size:16777216,withDup:false                                  1.75s   571.08m
HASH,Size:16777216,withDup:true                                   1.78s   563.18m

CC: @mbasmanova @Yuhta

wypb avatar Oct 22 '24 07:10 wypb

Deploy Preview for meta-velox canceled.

Name Link
Latest commit 183d3df7d6263af96e81b6babefa8e81d32a1934
Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/67175b2c95a6e20008177488

netlify[bot] avatar Oct 22 '24 07:10 netlify[bot]

This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the PR, make sure you've addressed reviewer comments, and rebase on the latest main. Thank you for your contributions!

stale[bot] avatar Jan 21 '25 06:01 stale[bot]