databend
databend copied to clipboard
feat: implement partial full outer join for merge into I/O improvement
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
PartitialFullOuter is an optimization for merge into source build, in this case, we can do pass the target block of matched partition and ummatched partition, and append block directly, but it's limitted, we just support the matched partial block must be modfied(update or delete), so we need to make sure the last condition for matched expression is none. for example:
a. merge into t1 using t2 on t1.a = t2.a when matched and cond1 then xx when matched then xxx.
b. merge into t1 using t2 on t1.a = t2.a when matched and cond1 then xx when matched and cond2 then xxx.
we can support a not b. The reason for this limitation is that if a target block is not modfied, we will append the unmacthed partial of target block unexpectedly.
we use this optimization to improve merge into I/O.
Tests
- [ ] Unit Test
- [x] Logic Test
- [ ] Benchmark Test
- [ ] No Test - Explain why
Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Breaking Change (fix or feature that could cause existing functionality not to work as expected)
- [ ] Documentation Update
- [ ] Refactoring
- [x] Performance Improvement
- [ ] Other (please describe):