doris
doris copied to clipboard
apache
[fix](Nereids) could not push down filter through cte producer sometimes
when consumer side eliminate some consumers from plan, the size of consumers is wrong. so we cannot push down some filter in producer side. this PR fix this problem by update consumer set after rewrite outer side
Issue Number: close #35379
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TPC-H: Total hot run time: 41921 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5af7e9df57d8e3a85af2e633cb3563244eb5ea1a, data reload: false
------ Round 1 ----------------------------------
q1 17619 4830 4257 4257
q2 2017 192 192 192
q3 10470 1169 1134 1134
q4 10199 803 816 803
q5 7468 2763 2674 2674
q6 222 132 128 128
q7 962 613 626 613
q8 9214 2154 2094 2094
q9 9624 6676 6709 6676
q10 9230 3892 3899 3892
q11 455 242 264 242
q12 427 228 225 225
q13 17458 3203 3236 3203
q14 252 216 219 216
q15 508 467 471 467
q16 505 407 422 407
q17 983 669 722 669
q18 8436 7770 7899 7770
q19 7072 1594 1541 1541
q20 654 310 326 310
q21 5208 4125 4159 4125
q22 370 283 291 283
Total cold run time: 119353 ms
Total hot run time: 41921 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4501 4427 4446 4427
q2 380 254 274 254
q3 3184 2938 2778 2778
q4 1887 1685 1664 1664
q5 5478 5503 5499 5499
q6 220 127 127 127
q7 2212 1855 1823 1823
q8 3296 3350 3398 3350
q9 8705 8701 8560 8560
q10 3940 3820 3836 3820
q11 616 523 506 506
q12 784 623 645 623
q13 16079 3157 3172 3157
q14 308 257 269 257
q15 524 476 480 476
q16 496 408 420 408
q17 1792 1488 1466 1466
q18 7687 7468 7359 7359
q19 1666 1596 1567 1567
q20 2018 1782 1782 1782
q21 7645 4877 4823 4823
q22 584 503 495 495
Total cold run time: 74002 ms
Total hot run time: 55221 ms
TPC-DS: Total hot run time: 170148 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5af7e9df57d8e3a85af2e633cb3563244eb5ea1a, data reload: false
query1 922 382 371 371
query2 6418 2535 2403 2403
query3 6631 207 206 206
query4 19929 17324 17268 17268
query5 4197 442 405 405
query6 250 157 159 157
query7 4581 308 288 288
query8 252 197 182 182
query9 8688 2438 2434 2434
query10 462 298 274 274
query11 10484 10191 10034 10034
query12 147 101 97 97
query13 1643 373 372 372
query14 9517 7564 7544 7544
query15 210 173 169 169
query16 7718 270 274 270
query17 1719 524 525 524
query18 1950 275 284 275
query19 197 153 161 153
query20 94 84 89 84
query21 198 130 133 130
query22 4231 3898 3889 3889
query23 33763 32933 33098 32933
query24 10522 2832 2846 2832
query25 568 354 368 354
query26 706 156 155 155
query27 2206 335 332 332
query28 5902 2102 2087 2087
query29 866 617 592 592
query30 250 147 153 147
query31 951 766 744 744
query32 90 73 60 60
query33 652 281 264 264
query34 866 501 483 483
query35 729 637 597 597
query36 1079 938 900 900
query37 102 66 68 66
query38 2896 2784 2770 2770
query39 827 795 790 790
query40 193 125 122 122
query41 45 41 44 41
query42 101 97 102 97
query43 591 525 575 525
query44 1103 725 733 725
query45 177 165 166 165
query46 1079 723 718 718
query47 1873 1769 1764 1764
query48 378 304 301 301
query49 842 379 382 379
query50 779 386 381 381
query51 6928 6769 6775 6769
query52 105 87 89 87
query53 344 299 287 287
query54 836 433 427 427
query55 75 75 75 75
query56 264 237 249 237
query57 1116 1041 1044 1041
query58 236 219 216 216
query59 3483 3204 3093 3093
query60 283 268 284 268
query61 93 84 89 84
query62 624 450 449 449
query63 310 281 287 281
query64 8427 2252 1783 1783
query65 3189 3111 3134 3111
query66 791 335 326 326
query67 15391 14852 14909 14852
query68 4537 540 534 534
query69 490 273 274 273
query70 1180 1107 1161 1107
query71 394 281 278 278
query72 7845 2743 2528 2528
query73 705 324 321 321
query74 6145 5598 5525 5525
query75 3444 2676 2608 2608
query76 3047 988 963 963
query77 579 267 269 267
query78 10239 9872 9906 9872
query79 2254 515 525 515
query80 923 447 434 434
query81 530 222 218 218
query82 714 92 92 92
query83 241 170 185 170
query84 246 90 86 86
query85 1459 271 265 265
query86 459 306 297 297
query87 3361 3100 3193 3100
query88 4243 2444 2427 2427
query89 478 398 378 378
query90 1969 190 190 190
query91 126 99 100 99
query92 57 50 49 49
query93 1916 519 494 494
query94 1193 183 196 183
query95 398 318 312 312
query96 589 269 268 268
query97 3144 3001 3053 3001
query98 245 219 223 219
query99 1191 937 836 836
Total cold run time: 267410 ms
Total hot run time: 170148 ms
ClickBench: Total hot run time: 31.07 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5af7e9df57d8e3a85af2e633cb3563244eb5ea1a, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.05
query3 0.23 0.05 0.05
query4 1.68 0.07 0.08
query5 0.49 0.49 0.50
query6 1.12 0.72 0.72
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.54 0.47 0.49
query10 0.55 0.55 0.53
query11 0.16 0.12 0.11
query12 0.14 0.13 0.12
query13 0.59 0.59 0.58
query14 0.77 0.78 0.79
query15 0.81 0.80 0.81
query16 0.36 0.36 0.37
query17 0.97 1.03 0.93
query18 0.23 0.23 0.24
query19 1.80 1.74 1.78
query20 0.01 0.01 0.01
query21 15.54 0.68 0.66
query22 4.72 5.97 2.71
query23 18.32 1.43 1.24
query24 2.02 0.23 0.19
query25 0.15 0.11 0.08
query26 0.27 0.16 0.16
query27 0.08 0.08 0.07
query28 13.28 1.00 0.99
query29 13.77 3.31 3.25
query30 0.24 0.05 0.06
query31 2.88 0.37 0.38
query32 3.31 0.46 0.47
query33 2.90 2.83 2.89
query34 17.08 4.41 4.40
query35 4.53 4.50 4.66
query36 0.66 0.46 0.45
query37 0.17 0.14 0.14
query38 0.15 0.15 0.14
query39 0.04 0.04 0.03
query40 0.15 0.14 0.13
query41 0.08 0.04 0.04
query42 0.05 0.05 0.04
query43 0.04 0.04 0.03
Total cold run time: 111.07 s
Total hot run time: 31.07 s
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
PR approved by at least one committer and no changes requested.