doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[improve](nereids) support pushdown count when scan project without any slot ref
Proposed changes
select 66 from baseall_dup;
could use pushAggOp=COUNT, so no need to scan real data
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TPC-DS: Total hot run time: 180938 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dfa428a809d9a98d7f93093b99af33c616e3b51c, data reload: false
query1 915 372 367 367
query2 6470 2420 2345 2345
query3 6656 216 211 211
query4 22959 21225 21176 21176
query5 4201 414 419 414
query6 273 168 178 168
query7 4591 285 282 282
query8 246 181 199 181
query9 8580 2368 2377 2368
query10 432 258 273 258
query11 14874 14301 14156 14156
query12 133 92 85 85
query13 1647 372 360 360
query14 9787 7078 8366 7078
query15 283 169 171 169
query16 8192 267 262 262
query17 1845 604 537 537
query18 2104 266 271 266
query19 213 152 147 147
query20 96 85 85 85
query21 193 131 129 129
query22 5075 4879 4878 4878
query23 34307 33481 33310 33310
query24 11120 2914 2956 2914
query25 624 369 366 366
query26 1506 158 151 151
query27 3023 316 330 316
query28 7549 2056 2022 2022
query29 920 627 614 614
query30 329 174 172 172
query31 974 749 717 717
query32 97 52 51 51
query33 744 246 246 246
query34 1039 476 490 476
query35 836 693 679 679
query36 1084 896 899 896
query37 140 71 73 71
query38 2898 2764 2747 2747
query39 1619 1584 1547 1547
query40 276 123 129 123
query41 46 43 42 42
query42 104 96 96 96
query43 579 550 526 526
query44 1249 718 742 718
query45 266 249 255 249
query46 1079 736 731 731
query47 1983 1874 1869 1869
query48 368 300 293 293
query49 1158 403 395 395
query50 775 383 389 383
query51 6844 6756 6758 6756
query52 102 90 94 90
query53 350 278 276 276
query54 839 424 432 424
query55 79 73 76 73
query56 245 224 218 218
query57 1257 1135 1164 1135
query58 219 194 201 194
query59 3262 3331 3374 3331
query60 249 239 232 232
query61 94 105 84 84
query62 672 481 468 468
query63 309 281 279 279
query64 9575 2202 1743 1743
query65 3162 3065 3109 3065
query66 1388 356 345 345
query67 15488 15149 15010 15010
query68 4570 527 537 527
query69 464 306 314 306
query70 1177 1134 1138 1134
query71 404 266 264 264
query72 8021 2591 2354 2354
query73 708 323 321 321
query74 6659 6180 6256 6180
query75 3358 2620 2630 2620
query76 2526 1048 1040 1040
query77 397 272 263 263
query78 10734 10080 10002 10002
query79 1941 511 514 511
query80 887 496 428 428
query81 527 249 241 241
query82 645 97 97 97
query83 237 167 167 167
query84 237 89 86 86
query85 1866 269 262 262
query86 498 295 311 295
query87 3284 3142 3105 3105
query88 4256 2343 2344 2343
query89 464 374 382 374
query90 1953 197 197 197
query91 125 101 97 97
query92 56 48 49 48
query93 2324 502 480 480
query94 1268 182 181 181
query95 392 299 354 299
query96 606 273 276 273
query97 3205 2992 2997 2992
query98 233 214 207 207
query99 1142 958 879 879
Total cold run time: 285755 ms
Total hot run time: 180938 ms
ClickBench: Total hot run time: 30.74 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dfa428a809d9a98d7f93093b99af33c616e3b51c, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.03 0.04
query3 0.23 0.06 0.05
query4 1.70 0.06 0.08
query5 0.50 0.48 0.50
query6 1.13 0.72 0.72
query7 0.02 0.01 0.02
query8 0.04 0.04 0.04
query9 0.55 0.48 0.50
query10 0.56 0.55 0.53
query11 0.14 0.12 0.11
query12 0.14 0.12 0.12
query13 0.59 0.59 0.60
query14 0.79 0.77 0.78
query15 0.84 0.82 0.82
query16 0.35 0.35 0.35
query17 0.95 0.96 0.97
query18 0.20 0.25 0.27
query19 1.76 1.70 1.70
query20 0.02 0.01 0.01
query21 15.45 0.72 0.66
query22 3.89 7.69 2.13
query23 18.31 1.42 1.33
query24 1.61 0.30 0.21
query25 0.15 0.08 0.08
query26 0.26 0.17 0.17
query27 0.08 0.07 0.08
query28 13.37 1.02 1.01
query29 12.71 3.24 3.21
query30 0.24 0.07 0.06
query31 2.87 0.40 0.38
query32 3.27 0.49 0.47
query33 2.92 2.91 2.94
query34 17.06 4.43 4.46
query35 4.50 4.47 4.49
query36 0.66 0.46 0.45
query37 0.18 0.15 0.16
query38 0.14 0.14 0.15
query39 0.05 0.04 0.04
query40 0.17 0.14 0.14
query41 0.09 0.05 0.04
query42 0.05 0.05 0.04
query43 0.04 0.04 0.04
Total cold run time: 108.7 s
Total hot run time: 30.74 s
TPC-DS: Total hot run time: 182104 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d07699fd9242e64899412600ed81ffedda173c5f, data reload: false
query1 915 383 370 370
query2 6737 2320 2385 2320
query3 6652 206 204 204
query4 24533 21244 21179 21179
query5 4148 424 417 417
query6 263 172 176 172
query7 4583 292 288 288
query8 239 188 190 188
query9 8455 2393 2388 2388
query10 435 258 246 246
query11 14671 14131 14128 14128
query12 141 91 88 88
query13 1638 378 371 371
query14 10624 8528 8316 8316
query15 260 166 177 166
query16 8129 259 253 253
query17 1860 558 532 532
query18 2087 269 269 269
query19 207 149 147 147
query20 98 87 88 87
query21 195 134 137 134
query22 5114 4847 4875 4847
query23 34307 33524 33616 33524
query24 11031 2946 2960 2946
query25 606 375 362 362
query26 1335 157 155 155
query27 2981 320 325 320
query28 7581 2040 2026 2026
query29 867 613 593 593
query30 276 176 184 176
query31 981 756 721 721
query32 91 52 51 51
query33 740 246 238 238
query34 1035 479 476 476
query35 835 667 676 667
query36 1096 851 929 851
query37 132 70 70 70
query38 2951 2775 2763 2763
query39 1625 1561 1595 1561
query40 204 123 125 123
query41 46 40 41 40
query42 105 93 95 93
query43 539 544 558 544
query44 1180 731 743 731
query45 257 251 269 251
query46 1070 778 721 721
query47 1954 1880 1908 1880
query48 360 291 293 291
query49 1073 398 389 389
query50 767 370 369 369
query51 6890 6722 6675 6675
query52 106 87 93 87
query53 347 280 279 279
query54 906 426 429 426
query55 77 95 70 70
query56 236 227 219 219
query57 1259 1135 1152 1135
query58 225 201 206 201
query59 3476 3020 3255 3020
query60 257 224 224 224
query61 95 89 86 86
query62 685 486 461 461
query63 315 282 279 279
query64 8955 2191 1757 1757
query65 3191 3132 3118 3118
query66 1395 337 343 337
query67 15711 15009 15232 15009
query68 4540 531 541 531
query69 479 292 298 292
query70 1177 1116 1062 1062
query71 434 265 260 260
query72 7792 2549 2395 2395
query73 708 314 319 314
query74 6708 6334 6368 6334
query75 3390 2637 2630 2630
query76 2643 955 962 955
query77 439 260 261 260
query78 10686 10167 10064 10064
query79 2512 508 516 508
query80 1034 423 431 423
query81 530 247 242 242
query82 733 96 96 96
query83 240 161 164 161
query84 238 84 85 84
query85 1542 332 334 332
query86 485 302 324 302
query87 3328 3117 3075 3075
query88 4279 2322 2350 2322
query89 468 382 384 382
query90 1966 185 189 185
query91 126 98 103 98
query92 63 49 49 49
query93 1689 517 493 493
query94 1219 174 184 174
query95 399 300 304 300
query96 572 270 266 266
query97 3196 2997 2990 2990
query98 232 214 216 214
query99 1134 892 886 886
Total cold run time: 287116 ms
Total hot run time: 182104 ms
ClickBench: Total hot run time: 30.47 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d07699fd9242e64899412600ed81ffedda173c5f, data reload: false
query1 0.04 0.03 0.03
query2 0.08 0.04 0.04
query3 0.23 0.06 0.05
query4 1.68 0.06 0.07
query5 0.49 0.49 0.52
query6 1.13 0.72 0.72
query7 0.02 0.02 0.02
query8 0.05 0.03 0.04
query9 0.54 0.49 0.48
query10 0.54 0.55 0.55
query11 0.14 0.11 0.11
query12 0.15 0.12 0.11
query13 0.59 0.59 0.61
query14 0.77 0.77 0.77
query15 0.83 0.82 0.82
query16 0.35 0.37 0.35
query17 0.98 0.94 1.03
query18 0.23 0.23 0.26
query19 1.85 1.68 1.74
query20 0.01 0.00 0.01
query21 15.50 0.67 0.64
query22 4.80 6.86 2.05
query23 18.32 1.33 1.23
query24 1.36 0.28 0.27
query25 0.15 0.09 0.08
query26 0.24 0.16 0.16
query27 0.08 0.09 0.08
query28 13.50 1.01 0.99
query29 12.95 3.24 3.24
query30 0.24 0.06 0.06
query31 2.86 0.39 0.37
query32 3.30 0.48 0.47
query33 2.91 2.86 2.96
query34 17.05 4.44 4.38
query35 4.48 4.47 4.57
query36 0.65 0.48 0.47
query37 0.18 0.15 0.16
query38 0.15 0.14 0.14
query39 0.04 0.03 0.03
query40 0.16 0.15 0.14
query41 0.09 0.04 0.05
query42 0.05 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 109.8 s
Total hot run time: 30.47 s
TPC-H: Total hot run time: 42089 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f1b6021c851c6f736c7795e011de8a212dac43fc, data reload: false
------ Round 1 ----------------------------------
q1 17610 4421 4292 4292
q2 2023 187 213 187
q3 10459 1251 1230 1230
q4 10198 866 845 845
q5 7512 2724 2764 2724
q6 221 133 136 133
q7 1036 589 612 589
q8 9243 2165 2135 2135
q9 9540 6771 6701 6701
q10 9676 3957 3894 3894
q11 464 250 237 237
q12 416 235 240 235
q13 17246 3210 3250 3210
q14 283 228 216 216
q15 508 472 471 471
q16 497 386 379 379
q17 1009 683 746 683
q18 8582 7803 7756 7756
q19 3411 1565 1542 1542
q20 669 316 312 312
q21 5270 4049 4121 4049
q22 356 290 269 269
Total cold run time: 116229 ms
Total hot run time: 42089 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4603 4457 4392 4392
q2 371 276 258 258
q3 3147 2940 2765 2765
q4 1872 1613 1664 1613
q5 5503 5538 5507 5507
q6 213 125 130 125
q7 2365 2007 2026 2007
q8 3266 3399 3466 3399
q9 8725 8755 8616 8616
q10 3949 3845 3879 3845
q11 608 490 497 490
q12 801 623 660 623
q13 17068 3236 3143 3143
q14 294 263 256 256
q15 528 475 465 465
q16 466 423 415 415
q17 1770 1488 1462 1462
q18 7844 7747 7428 7428
q19 1718 1571 1544 1544
q20 2021 1784 1769 1769
q21 8972 4923 4790 4790
q22 575 465 493 465
Total cold run time: 76679 ms
Total hot run time: 55377 ms
TPC-DS: Total hot run time: 180484 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f1b6021c851c6f736c7795e011de8a212dac43fc, data reload: false
query1 906 385 372 372
query2 6875 2396 2232 2232
query3 6648 207 205 205
query4 22735 21235 21238 21235
query5 4183 415 411 411
query6 262 184 175 175
query7 4580 287 288 287
query8 239 186 198 186
query9 8541 2393 2373 2373
query10 444 254 255 254
query11 14743 14139 14081 14081
query12 140 92 91 91
query13 1660 375 371 371
query14 10049 8342 6848 6848
query15 214 170 174 170
query16 7935 266 250 250
query17 1782 561 540 540
query18 2029 271 263 263
query19 203 146 152 146
query20 97 85 83 83
query21 203 131 133 131
query22 5009 4856 4902 4856
query23 34283 33586 33603 33586
query24 6753 2871 2937 2871
query25 525 365 365 365
query26 709 156 160 156
query27 1892 313 312 312
query28 3642 2052 2051 2051
query29 862 606 596 596
query30 244 178 181 178
query31 995 751 736 736
query32 90 52 52 52
query33 487 250 239 239
query34 866 470 462 462
query35 771 669 659 659
query36 1026 870 896 870
query37 112 70 70 70
query38 2899 2773 2742 2742
query39 1622 1566 1555 1555
query40 196 131 128 128
query41 46 45 45 45
query42 103 98 98 98
query43 567 544 533 533
query44 1160 729 740 729
query45 262 251 252 251
query46 1065 700 700 700
query47 1975 1871 1877 1871
query48 380 300 297 297
query49 774 390 424 390
query50 775 374 387 374
query51 6884 6816 6851 6816
query52 102 95 90 90
query53 352 279 289 279
query54 516 427 434 427
query55 80 72 70 70
query56 240 216 222 216
query57 1253 1174 1182 1174
query58 223 194 194 194
query59 3408 3261 3401 3261
query60 269 224 246 224
query61 93 86 85 85
query62 600 456 472 456
query63 308 290 288 288
query64 8422 2189 1710 1710
query65 3189 3122 3114 3114
query66 792 344 334 334
query67 15280 14941 14769 14769
query68 4537 544 534 534
query69 484 314 307 307
query70 1133 1099 1060 1060
query71 396 325 262 262
query72 7989 2574 2341 2341
query73 713 321 322 321
query74 6674 6234 6263 6234
query75 3282 2641 2599 2599
query76 2321 974 957 957
query77 418 266 269 266
query78 10768 10240 10070 10070
query79 2096 510 525 510
query80 1022 434 434 434
query81 528 240 244 240
query82 951 96 95 95
query83 250 165 163 163
query84 243 87 86 86
query85 1024 328 267 267
query86 450 295 300 295
query87 3280 3091 3073 3073
query88 3431 2317 2328 2317
query89 475 375 386 375
query90 2027 187 195 187
query91 122 97 96 96
query92 58 52 49 49
query93 2067 508 497 497
query94 1163 182 185 182
query95 394 300 299 299
query96 590 265 263 263
query97 3152 3024 3060 3024
query98 249 217 210 210
query99 1224 888 904 888
Total cold run time: 270080 ms
Total hot run time: 180484 ms
ClickBench: Total hot run time: 30.8 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f1b6021c851c6f736c7795e011de8a212dac43fc, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.69 0.08 0.07
query5 0.49 0.49 0.50
query6 1.12 0.74 0.72
query7 0.02 0.01 0.01
query8 0.05 0.04 0.04
query9 0.54 0.50 0.49
query10 0.53 0.55 0.54
query11 0.15 0.11 0.11
query12 0.15 0.11 0.13
query13 0.59 0.58 0.59
query14 0.78 0.79 0.76
query15 0.82 0.81 0.80
query16 0.35 0.38 0.37
query17 1.02 1.01 0.94
query18 0.20 0.25 0.25
query19 1.89 1.74 1.65
query20 0.01 0.01 0.01
query21 15.57 0.68 0.65
query22 4.64 6.66 2.22
query23 18.32 1.33 1.21
query24 1.72 0.30 0.21
query25 0.14 0.08 0.08
query26 0.26 0.17 0.16
query27 0.08 0.07 0.08
query28 13.34 1.02 1.00
query29 13.73 3.31 3.31
query30 0.24 0.06 0.06
query31 2.86 0.39 0.38
query32 3.26 0.47 0.47
query33 2.84 2.93 2.90
query34 17.14 4.45 4.49
query35 4.52 4.52 4.65
query36 0.66 0.48 0.47
query37 0.18 0.15 0.15
query38 0.15 0.14 0.15
query39 0.04 0.03 0.03
query40 0.17 0.14 0.16
query41 0.09 0.05 0.04
query42 0.06 0.04 0.04
query43 0.04 0.04 0.04
Total cold run time: 110.8 s
Total hot run time: 30.8 s
TPC-DS: Total hot run time: 171453 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 509efa63ae33ac32de8d853ebc7c5156d1a5e3e2, data reload: false
query1 910 375 375 375
query2 6466 2452 2278 2278
query3 6662 203 206 203
query4 19165 17396 17199 17199
query5 4179 423 404 404
query6 249 160 157 157
query7 4590 287 286 286
query8 243 197 179 179
query9 8588 2370 2359 2359
query10 436 267 251 251
query11 10567 10081 10129 10081
query12 133 89 85 85
query13 1638 351 373 351
query14 9211 7162 6887 6887
query15 218 174 177 174
query16 7903 256 258 256
query17 1860 523 515 515
query18 1967 269 268 268
query19 208 177 152 152
query20 94 86 83 83
query21 193 128 124 124
query22 4190 4143 3901 3901
query23 33480 33012 32945 32945
query24 11109 2855 2807 2807
query25 634 347 359 347
query26 1429 159 159 159
query27 2995 326 328 326
query28 7598 2027 2044 2027
query29 950 597 600 597
query30 311 175 176 175
query31 965 767 715 715
query32 95 52 57 52
query33 752 264 280 264
query34 997 464 481 464
query35 728 613 580 580
query36 1050 878 907 878
query37 144 71 69 69
query38 2913 2792 2765 2765
query39 856 796 802 796
query40 206 125 125 125
query41 44 42 42 42
query42 102 94 92 92
query43 604 549 568 549
query44 1200 724 741 724
query45 177 161 163 161
query46 1080 704 701 701
query47 1871 1781 1755 1755
query48 363 296 288 288
query49 1073 366 375 366
query50 771 388 403 388
query51 6840 6876 6706 6706
query52 102 91 90 90
query53 354 302 286 286
query54 914 427 427 427
query55 75 74 70 70
query56 254 234 244 234
query57 1161 1039 1038 1038
query58 226 199 205 199
query59 3409 3242 3114 3114
query60 269 241 247 241
query61 90 89 88 88
query62 648 446 465 446
query63 307 285 281 281
query64 9955 2156 1716 1716
query65 3164 3107 3122 3107
query66 1394 331 315 315
query67 15339 14860 14776 14776
query68 4568 525 537 525
query69 468 265 261 261
query70 1140 1107 1067 1067
query71 421 263 279 263
query72 7601 5851 5662 5662
query73 717 316 315 315
query74 6102 5605 5604 5604
query75 3589 2647 2598 2598
query76 3026 984 997 984
query77 643 258 262 258
query78 10244 9755 9600 9600
query79 2086 510 511 510
query80 744 426 426 426
query81 519 241 245 241
query82 1344 92 92 92
query83 194 175 174 174
query84 239 89 84 84
query85 1538 273 308 273
query86 468 304 271 271
query87 3251 3094 3165 3094
query88 4063 2341 2315 2315
query89 470 389 380 380
query90 1986 185 182 182
query91 131 107 107 107
query92 63 50 48 48
query93 2014 517 503 503
query94 1222 264 182 182
query95 403 303 298 298
query96 583 261 263 261
query97 3139 2959 3011 2959
query98 240 226 220 220
query99 1162 854 837 837
Total cold run time: 273046 ms
Total hot run time: 171453 ms
ClickBench: Total hot run time: 30.36 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 509efa63ae33ac32de8d853ebc7c5156d1a5e3e2, data reload: false
query1 0.04 0.03 0.04
query2 0.08 0.04 0.04
query3 0.23 0.05 0.05
query4 1.69 0.07 0.08
query5 0.49 0.49 0.53
query6 1.12 0.72 0.72
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.55 0.48 0.49
query10 0.54 0.54 0.54
query11 0.17 0.11 0.12
query12 0.14 0.12 0.12
query13 0.60 0.58 0.61
query14 0.76 0.79 0.77
query15 0.83 0.81 0.81
query16 0.36 0.37 0.36
query17 0.97 0.93 0.92
query18 0.23 0.23 0.22
query19 1.76 1.75 1.80
query20 0.02 0.01 0.01
query21 15.70 0.66 0.65
query22 3.89 8.05 1.78
query23 18.26 1.41 1.22
query24 2.13 0.20 0.22
query25 0.16 0.08 0.08
query26 0.25 0.16 0.16
query27 0.08 0.08 0.07
query28 13.23 1.09 1.06
query29 12.56 3.29 3.25
query30 0.25 0.06 0.05
query31 2.86 0.39 0.37
query32 3.30 0.48 0.47
query33 2.91 2.88 2.91
query34 17.02 4.44 4.45
query35 4.49 4.53 4.51
query36 0.65 0.49 0.46
query37 0.17 0.16 0.15
query38 0.16 0.14 0.15
query39 0.05 0.03 0.03
query40 0.16 0.14 0.15
query41 0.10 0.04 0.05
query42 0.05 0.05 0.05
query43 0.04 0.04 0.04
Total cold run time: 109.12 s
Total hot run time: 30.36 s