tidb icon indicating copy to clipboard operation
tidb copied to clipboard

executor: Refine partition number in hash join v2

Open windtalker opened this issue 7 months ago • 3 comments

What problem does this PR solve?

Issue Number: ref #53127

Problem Summary: Current, we dispatch row to each partition by the LSB bits in hash value https://github.com/pingcap/tidb/blob/31c1de0b4f30d708de8228aa72c1a0dbff547504/pkg/executor/join/join_row_table.go#L540 And in each partition, we build hash table also based on the LSB bits in hash value https://github.com/pingcap/tidb/blob/31c1de0b4f30d708de8228aa72c1a0dbff547504/pkg/executor/join/hash_table_v2.go#L86 This may increase the hash table conflicts especially if partition number is power of 2

What changed and how does it work?

  1. make the partition number to be a number that is pow of 2
  2. dispatch row to each partition by the n MSB bits in hash value

Check List

Tests

  • [x] Unit test
  • [ ] Integration test
  • [ ] Manual test (add detailed scripts or steps below)
  • [ ] No need to test
    • [ ] I checked and no code files have been changed.

Side effects

  • [ ] Performance regression: Consumes more CPU
  • [ ] Performance regression: Consumes more Memory
  • [ ] Breaking backward compatibility

Documentation

  • [ ] Affects user behaviors
  • [ ] Contains syntax changes
  • [ ] Contains variable changes
  • [ ] Contains experimental features
  • [ ] Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

windtalker avatar Jun 28 '24 06:06 windtalker