doris
doris copied to clipboard
[feature](Nereids): add rule split limit into two phase
Proposed changes
- Add a rule split limit, like Limit(Origin) ==> Limit(Global) -> Gather -> Limit(Local)
- Add a rule: limit-> sort ====> topN
- fix a bug about topN
- make the type of limit,offset long in topN
And because this rule is always beneficial, we add a rule in the rewrite phase
Problem summary
Describe your changes.
Checklist(Required)
- [ ] Does it affect the original behavior
- [x] Has unit tests been added
- [ ] Has document been added or modified
- [ ] Does it need to update dependencies
- [ ] Is this PR support rollback (If NO, please explain WHY)
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
TeamCity pipeline, clickbench performance test result: the sum of best hot time: 33.76 seconds stream load tsv: 480 seconds loaded 74807831229 Bytes, about 148 MB/s stream load json: 40 seconds loaded 2358488459 Bytes, about 56 MB/s stream load orc: 74 seconds loaded 1101869774 Bytes, about 14 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230307060535_clickbench_pr_109806.html
run buildall
run buildall
run buildall
run buildall
run buildall
PR approved by at least one committer and no changes requested.
PR approved by anyone and no changes requested.