doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[Enhancement] (nereids) optimize CleanQueryStatsCommand and remove CleanQueryStatsStmt
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
-
Test
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
-
Behavior changed:
- [ ] No.
- [ ] Yes.
-
Does this need documentation?
- [ ] No.
- [ ] Yes.
Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:
- What problem was fixed (it's best to include specific error reporting information). How it was fixed.
- Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
- What features were added. Why was this function added?
- Which code was refactored and why was this part of the code refactored?
- Which functions were optimized and what is the difference before and after the optimization?
TPC-H: Total hot run time: 33831 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0d01aa397b9198346b38b3d1adcf639d53516d43, data reload: false
------ Round 1 ----------------------------------
q1 17601 5138 5013 5013
q2 1951 290 196 196
q3 10267 1294 694 694
q4 10231 1020 517 517
q5 7535 2326 2367 2326
q6 180 163 132 132
q7 900 737 622 622
q8 9314 1290 1069 1069
q9 6790 5082 5164 5082
q10 6898 2395 1953 1953
q11 483 293 311 293
q12 343 350 214 214
q13 17761 3637 3056 3056
q14 233 240 212 212
q15 546 507 490 490
q16 427 423 384 384
q17 597 857 364 364
q18 7947 7270 7136 7136
q19 1642 967 554 554
q20 327 346 224 224
q21 3619 3139 2315 2315
q22 1074 1027 985 985
Total cold run time: 106666 ms
Total hot run time: 33831 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5089 5048 5077 5048
q2 242 320 226 226
q3 2190 2647 2289 2289
q4 1333 1751 1396 1396
q5 4240 4118 4438 4118
q6 211 175 129 129
q7 2074 1916 1744 1744
q8 2626 2561 2501 2501
q9 7243 7168 7211 7168
q10 3095 3261 2800 2800
q11 601 516 495 495
q12 701 779 602 602
q13 3504 3926 3356 3356
q14 284 289 258 258
q15 553 493 494 493
q16 447 492 426 426
q17 1139 1521 1368 1368
q18 7746 7406 7523 7406
q19 809 845 839 839
q20 1949 2056 1867 1867
q21 4887 4412 4295 4295
q22 1046 1027 994 994
Total cold run time: 52009 ms
Total hot run time: 49818 ms
TPC-DS: Total hot run time: 185779 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0d01aa397b9198346b38b3d1adcf639d53516d43, data reload: false
query1 982 393 384 384
query2 6540 1859 1822 1822
query3 6748 225 213 213
query4 26694 23439 23689 23439
query5 4823 637 468 468
query6 315 206 199 199
query7 4628 492 286 286
query8 261 228 243 228
query9 8622 2677 2689 2677
query10 526 362 282 282
query11 15563 15074 14824 14824
query12 161 119 106 106
query13 1635 526 392 392
query14 9045 5939 6071 5939
query15 205 191 167 167
query16 7150 616 500 500
query17 1163 717 555 555
query18 1973 399 304 304
query19 189 178 156 156
query20 128 118 112 112
query21 215 123 114 114
query22 4012 4162 3913 3913
query23 33920 33142 33151 33142
query24 8434 2350 2386 2350
query25 552 452 425 425
query26 1246 280 156 156
query27 2756 522 357 357
query28 4375 2178 2149 2149
query29 787 603 461 461
query30 292 224 201 201
query31 918 850 781 781
query32 80 63 70 63
query33 571 375 311 311
query34 791 860 538 538
query35 789 835 731 731
query36 946 988 862 862
query37 119 104 82 82
query38 4080 4063 4011 4011
query39 1495 1431 1407 1407
query40 215 127 116 116
query41 71 65 67 65
query42 132 114 115 114
query43 509 497 487 487
query44 1298 825 858 825
query45 177 179 164 164
query46 837 1025 630 630
query47 1722 1777 1745 1745
query48 389 419 306 306
query49 742 494 411 411
query50 652 674 448 448
query51 4118 4199 4102 4102
query52 109 116 99 99
query53 227 262 183 183
query54 575 572 509 509
query55 87 82 83 82
query56 296 300 291 291
query57 1176 1201 1108 1108
query58 268 257 256 256
query59 2691 2701 2600 2600
query60 338 333 305 305
query61 123 124 126 124
query62 805 757 650 650
query63 225 188 188 188
query64 4356 1032 674 674
query65 4293 4182 4176 4176
query66 1140 414 318 318
query67 16019 15506 15154 15154
query68 8952 888 533 533
query69 488 307 268 268
query70 1203 1081 1109 1081
query71 464 316 298 298
query72 5367 4742 4787 4742
query73 727 641 349 349
query74 8809 9179 8934 8934
query75 4242 3203 2684 2684
query76 3656 1174 741 741
query77 846 376 296 296
query78 10115 10290 9357 9357
query79 1878 836 585 585
query80 584 520 447 447
query81 496 266 230 230
query82 454 131 101 101
query83 251 251 252 251
query84 249 113 89 89
query85 776 360 354 354
query86 385 292 300 292
query87 4341 4370 4416 4370
query88 3411 2313 2292 2292
query89 405 315 282 282
query90 1895 220 221 220
query91 150 161 116 116
query92 80 61 57 57
query93 1562 961 595 595
query94 685 427 309 309
query95 391 301 287 287
query96 497 596 283 283
query97 2715 2736 2612 2612
query98 237 208 209 208
query99 1460 1427 1279 1279
Total cold run time: 275090 ms
Total hot run time: 185779 ms
ClickBench: Total hot run time: 29.39 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0d01aa397b9198346b38b3d1adcf639d53516d43, data reload: false
query1 0.04 0.04 0.03
query2 0.07 0.04 0.03
query3 0.23 0.07 0.06
query4 1.61 0.10 0.10
query5 0.42 0.43 0.41
query6 1.15 0.65 0.66
query7 0.03 0.01 0.02
query8 0.04 0.04 0.03
query9 0.57 0.52 0.51
query10 0.57 0.59 0.56
query11 0.16 0.12 0.11
query12 0.15 0.12 0.12
query13 0.64 0.61 0.61
query14 0.80 0.82 0.82
query15 0.90 0.85 0.89
query16 0.39 0.38 0.40
query17 1.12 1.10 1.09
query18 0.23 0.21 0.21
query19 1.95 1.82 1.83
query20 0.02 0.01 0.01
query21 15.40 0.91 0.56
query22 0.76 1.26 0.65
query23 14.87 1.37 0.63
query24 6.66 1.74 0.60
query25 0.50 0.14 0.07
query26 0.66 0.16 0.14
query27 0.06 0.05 0.05
query28 9.54 0.90 0.43
query29 12.52 4.14 3.44
query30 0.26 0.09 0.07
query31 2.82 0.60 0.39
query32 3.23 0.56 0.48
query33 3.06 3.12 3.05
query34 16.08 5.36 4.78
query35 4.81 4.85 4.82
query36 0.68 0.50 0.49
query37 0.09 0.07 0.07
query38 0.05 0.04 0.03
query39 0.03 0.03 0.02
query40 0.17 0.14 0.14
query41 0.07 0.02 0.03
query42 0.03 0.03 0.02
query43 0.03 0.03 0.03
Total cold run time: 103.47 s
Total hot run time: 29.39 s
TPC-H: Total hot run time: 33837 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 11477e7eb5df13f299fcb6b3647db8ed8ab66e5c, data reload: false
------ Round 1 ----------------------------------
q1 17629 5232 5072 5072
q2 1944 296 212 212
q3 10265 1340 705 705
q4 10213 1013 523 523
q5 7523 2346 2291 2291
q6 177 162 129 129
q7 880 747 624 624
q8 9297 1218 1041 1041
q9 6702 5072 5096 5072
q10 6913 2373 1959 1959
q11 496 290 265 265
q12 351 344 214 214
q13 17772 3684 3127 3127
q14 216 225 221 221
q15 568 493 494 493
q16 426 429 369 369
q17 577 849 365 365
q18 7431 7064 7079 7064
q19 2471 960 561 561
q20 335 330 220 220
q21 3659 2521 2373 2373
q22 1043 1021 937 937
Total cold run time: 106888 ms
Total hot run time: 33837 ms
----- Round 2, with runtime_filter_mode=off -----
q1 5195 5047 5038 5038
q2 237 315 225 225
q3 2200 2702 2313 2313
q4 1370 1780 1355 1355
q5 4220 4090 4272 4090
q6 208 169 126 126
q7 2045 1996 1763 1763
q8 2533 2616 2506 2506
q9 7064 7022 7023 7022
q10 3065 3254 2848 2848
q11 588 501 493 493
q12 701 775 624 624
q13 3506 3924 3280 3280
q14 290 290 269 269
q15 508 480 487 480
q16 433 477 429 429
q17 1143 1537 1384 1384
q18 7393 7032 7023 7023
q19 775 777 930 777
q20 1867 1956 1876 1876
q21 4644 4280 4265 4265
q22 1066 1013 993 993
Total cold run time: 51051 ms
Total hot run time: 49179 ms
TPC-DS: Total hot run time: 185996 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 11477e7eb5df13f299fcb6b3647db8ed8ab66e5c, data reload: false
query1 986 382 377 377
query2 6509 1886 1873 1873
query3 6741 230 218 218
query4 26213 23380 23582 23380
query5 4387 604 476 476
query6 311 206 201 201
query7 4624 497 302 302
query8 274 235 223 223
query9 8605 2608 2627 2608
query10 479 333 266 266
query11 15892 15056 15039 15039
query12 170 110 115 110
query13 1654 529 409 409
query14 9754 6168 6078 6078
query15 200 186 172 172
query16 7652 646 460 460
query17 1210 716 547 547
query18 2012 398 301 301
query19 198 182 164 164
query20 119 116 134 116
query21 210 120 108 108
query22 4068 4114 4060 4060
query23 33887 32860 33143 32860
query24 8415 2389 2391 2389
query25 501 442 393 393
query26 910 271 150 150
query27 2735 511 340 340
query28 4274 2127 2112 2112
query29 668 577 447 447
query30 284 212 193 193
query31 947 850 728 728
query32 70 66 65 65
query33 545 368 337 337
query34 795 854 535 535
query35 810 838 724 724
query36 938 973 884 884
query37 108 101 74 74
query38 4152 4147 4112 4112
query39 1458 1442 1409 1409
query40 210 116 104 104
query41 61 58 58 58
query42 126 106 106 106
query43 518 531 488 488
query44 1314 834 829 829
query45 183 177 166 166
query46 846 1003 619 619
query47 1737 1790 1711 1711
query48 378 424 305 305
query49 708 480 402 402
query50 653 666 415 415
query51 4118 4090 4069 4069
query52 117 108 100 100
query53 227 255 183 183
query54 567 559 518 518
query55 84 80 81 80
query56 304 301 291 291
query57 1160 1196 1125 1125
query58 261 266 244 244
query59 2705 2732 2655 2655
query60 331 317 297 297
query61 129 119 119 119
query62 812 803 651 651
query63 228 192 186 186
query64 3508 1097 774 774
query65 4254 4173 4179 4173
query66 904 392 314 314
query67 15658 15618 15211 15211
query68 8477 899 529 529
query69 461 305 275 275
query70 1252 1064 1103 1064
query71 454 320 306 306
query72 5672 4643 4707 4643
query73 713 597 349 349
query74 8913 9149 8981 8981
query75 3889 3185 2698 2698
query76 3596 1196 754 754
query77 793 382 286 286
query78 10165 10161 9391 9391
query79 2137 817 587 587
query80 600 502 441 441
query81 499 266 229 229
query82 453 126 96 96
query83 250 240 242 240
query84 243 110 97 97
query85 811 350 318 318
query86 385 314 277 277
query87 4391 4445 4415 4415
query88 3609 2309 2376 2309
query89 373 316 293 293
query90 1841 206 206 206
query91 139 139 111 111
query92 76 58 57 57
query93 1617 959 607 607
query94 758 410 284 284
query95 371 287 291 287
query96 484 564 281 281
query97 2702 2785 2648 2648
query98 242 208 205 205
query99 1667 1382 1299 1299
Total cold run time: 273899 ms
Total hot run time: 185996 ms
ClickBench: Total hot run time: 29.47 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 11477e7eb5df13f299fcb6b3647db8ed8ab66e5c, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.04 0.03
query3 0.24 0.06 0.07
query4 1.61 0.10 0.10
query5 0.42 0.45 0.42
query6 1.17 0.65 0.66
query7 0.03 0.01 0.02
query8 0.05 0.04 0.04
query9 0.58 0.51 0.53
query10 0.57 0.57 0.56
query11 0.16 0.10 0.11
query12 0.14 0.11 0.12
query13 0.63 0.61 0.61
query14 0.80 0.83 0.82
query15 0.91 0.88 0.90
query16 0.39 0.38 0.39
query17 1.04 1.08 1.05
query18 0.23 0.21 0.22
query19 2.02 1.87 1.87
query20 0.01 0.01 0.01
query21 15.40 0.94 0.55
query22 0.76 1.29 0.75
query23 14.76 1.40 0.61
query24 7.52 1.02 0.46
query25 0.55 0.22 0.07
query26 0.57 0.17 0.15
query27 0.05 0.05 0.05
query28 9.13 0.89 0.45
query29 12.54 4.10 3.36
query30 0.25 0.09 0.07
query31 2.83 0.61 0.41
query32 3.22 0.56 0.48
query33 3.03 3.15 3.12
query34 16.18 5.45 4.83
query35 4.84 4.91 4.85
query36 0.69 0.51 0.49
query37 0.09 0.06 0.06
query38 0.05 0.04 0.03
query39 0.03 0.03 0.02
query40 0.18 0.14 0.14
query41 0.08 0.02 0.02
query42 0.03 0.03 0.02
query43 0.04 0.03 0.03
Total cold run time: 103.93 s
Total hot run time: 29.47 s
PR approved by at least one committer and no changes requested.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}