doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[improvement](mtmv) Cache materialization check result for performance
Proposed changes
Need check materialization sql pattern in different abstract rule when rewrite by materialized view.
Such as the subClass of AbstractMaterializedViewJoinRule, MaterializedViewScanRule, AbstractMaterializedViewAggregateRule.
This check result can be cached when has checked, this can avoid unnecessary repeat check
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TPC-DS: Total hot run time: 187638 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de1a04cb83d9003d0ae6ac5c026288332769ace6, data reload: false
query1 908 354 343 343
query2 7534 2281 2429 2281
query3 6654 203 203 203
query4 24629 21685 21702 21685
query5 4109 426 417 417
query6 288 183 165 165
query7 4698 288 294 288
query8 228 184 195 184
query9 8406 2293 2294 2293
query10 420 252 246 246
query11 15407 14903 14730 14730
query12 118 87 83 83
query13 1650 363 384 363
query14 10598 8444 8435 8435
query15 278 177 177 177
query16 8352 264 256 256
query17 2083 599 570 570
query18 2125 280 283 280
query19 320 153 148 148
query20 87 84 83 83
query21 193 123 118 118
query22 4994 4942 4794 4794
query23 33939 33370 33183 33183
query24 10547 2780 2926 2780
query25 586 362 353 353
query26 1252 150 145 145
query27 2886 306 303 303
query28 7181 1926 1908 1908
query29 856 603 590 590
query30 247 150 149 149
query31 948 725 720 720
query32 89 52 52 52
query33 735 234 236 234
query34 1047 467 468 467
query35 818 686 647 647
query36 1087 905 906 905
query37 131 64 65 64
query38 3123 3039 3033 3033
query39 1581 1529 1538 1529
query40 193 124 122 122
query41 40 38 38 38
query42 109 95 97 95
query43 552 530 511 511
query44 1236 723 741 723
query45 255 257 242 242
query46 1072 719 689 689
query47 1950 1862 1879 1862
query48 364 296 294 294
query49 859 390 397 390
query50 752 376 374 374
query51 6846 6768 6688 6688
query52 102 89 90 89
query53 350 274 276 274
query54 314 237 231 231
query55 78 70 70 70
query56 250 232 224 224
query57 1200 1140 1160 1140
query58 220 194 197 194
query59 3399 3194 3127 3127
query60 261 227 231 227
query61 88 84 85 84
query62 628 460 434 434
query63 305 292 279 279
query64 8447 7146 7124 7124
query65 3122 3080 2993 2993
query66 1392 332 322 322
query67 15690 15113 15420 15113
query68 5161 511 519 511
query69 499 305 313 305
query70 1207 1104 1111 1104
query71 422 272 262 262
query72 7223 2534 2369 2369
query73 697 315 313 313
query74 6536 6170 6106 6106
query75 3397 2628 2586 2586
query76 3158 876 991 876
query77 361 266 260 260
query78 10893 10521 10275 10275
query79 7975 512 504 504
query80 1727 433 434 433
query81 521 225 219 219
query82 882 90 92 90
query83 205 169 173 169
query84 270 88 85 85
query85 1399 317 289 289
query86 453 308 312 308
query87 3212 3110 3131 3110
query88 5407 2422 2423 2422
query89 495 381 370 370
query90 2055 181 180 180
query91 125 97 97 97
query92 66 47 47 47
query93 6299 491 493 491
query94 1138 176 177 176
query95 392 292 300 292
query96 604 278 263 263
query97 3078 2897 2954 2897
query98 240 213 222 213
query99 1251 859 879 859
Total cold run time: 298554 ms
Total hot run time: 187638 ms
PR approved by at least one committer and no changes requested.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TPC-H: Total hot run time: 41679 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e16c150af649d8344ae162ca496980b10e2cf259, data reload: false
------ Round 1 ----------------------------------
q1 17620 4280 4213 4213
q2 2009 190 191 190
q3 10570 1196 1208 1196
q4 10191 775 795 775
q5 7463 2718 2697 2697
q6 215 140 135 135
q7 957 555 540 540
q8 9221 2113 2095 2095
q9 9390 6710 6691 6691
q10 9304 3849 3851 3849
q11 463 253 233 233
q12 494 219 224 219
q13 17415 3166 3278 3166
q14 252 213 208 208
q15 502 467 468 467
q16 512 402 395 395
q17 968 690 716 690
q18 8337 7836 7718 7718
q19 6122 1495 1538 1495
q20 647 320 324 320
q21 5228 4271 4103 4103
q22 343 284 285 284
Total cold run time: 118223 ms
Total hot run time: 41679 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4563 4410 4453 4410
q2 385 273 265 265
q3 3114 2988 2786 2786
q4 1864 1609 1593 1593
q5 5511 5487 5438 5438
q6 212 120 122 120
q7 1766 1406 1394 1394
q8 3250 3410 3648 3410
q9 8605 8622 8603 8603
q10 4022 3707 3822 3707
q11 594 498 493 493
q12 796 604 637 604
q13 16492 3093 3165 3093
q14 292 267 266 266
q15 538 487 471 471
q16 495 444 436 436
q17 1828 1527 1457 1457
q18 7658 7511 7439 7439
q19 1674 1515 1585 1515
q20 1987 1773 1765 1765
q21 8151 4771 4756 4756
q22 566 509 492 492
Total cold run time: 74363 ms
Total hot run time: 54513 ms
TPC-DS: Total hot run time: 187611 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e16c150af649d8344ae162ca496980b10e2cf259, data reload: false
query1 913 369 336 336
query2 6442 2458 2379 2379
query3 6631 206 212 206
query4 23617 21367 21318 21318
query5 4115 407 434 407
query6 273 174 186 174
query7 4573 288 280 280
query8 238 182 182 182
query9 8450 2417 2395 2395
query10 439 246 258 246
query11 14840 14251 14184 14184
query12 136 88 85 85
query13 1654 362 365 362
query14 9081 8485 8272 8272
query15 268 171 176 171
query16 8101 263 260 260
query17 1764 545 537 537
query18 2092 272 263 263
query19 206 146 147 146
query20 90 86 81 81
query21 193 126 125 125
query22 5023 4840 4851 4840
query23 34083 33485 33470 33470
query24 9014 2899 2829 2829
query25 561 358 360 358
query26 685 149 150 149
query27 2187 316 329 316
query28 5799 2052 2044 2044
query29 880 606 595 595
query30 225 158 152 152
query31 976 746 769 746
query32 92 52 59 52
query33 601 246 250 246
query34 865 483 480 480
query35 772 679 670 670
query36 1041 874 908 874
query37 101 68 65 65
query38 2855 2783 2768 2768
query39 1606 1558 1534 1534
query40 198 124 123 123
query41 46 37 38 37
query42 105 101 96 96
query43 569 535 540 535
query44 1040 740 725 725
query45 270 270 249 249
query46 1065 705 701 701
query47 1960 1894 1899 1894
query48 380 290 286 286
query49 834 389 381 381
query50 777 383 390 383
query51 6942 6795 6795 6795
query52 106 85 90 85
query53 354 273 278 273
query54 701 424 433 424
query55 74 69 71 69
query56 237 218 226 218
query57 1212 1157 1135 1135
query58 209 204 218 204
query59 3403 3247 3292 3247
query60 280 242 266 242
query61 90 86 89 86
query62 644 463 464 463
query63 313 282 282 282
query64 8396 7375 7360 7360
query65 3166 3082 3079 3079
query66 778 334 343 334
query67 15169 15018 15032 15018
query68 4572 535 544 535
query69 474 307 298 298
query70 1112 1128 1102 1102
query71 351 275 259 259
query72 7193 2607 2343 2343
query73 707 324 322 322
query74 6499 6023 6120 6023
query75 3299 2631 2597 2597
query76 2356 1030 990 990
query77 408 262 258 258
query78 10980 9990 9930 9930
query79 2366 503 515 503
query80 903 432 433 432
query81 523 223 215 215
query82 679 91 98 91
query83 230 161 206 161
query84 254 81 86 81
query85 1451 266 259 259
query86 477 320 320 320
query87 3244 3119 3112 3112
query88 4058 2416 2422 2416
query89 474 377 370 370
query90 1958 185 182 182
query91 125 93 95 93
query92 58 47 48 47
query93 1721 501 494 494
query94 1247 183 187 183
query95 393 298 301 298
query96 587 275 265 265
query97 3176 2992 2994 2992
query98 248 215 218 215
query99 1191 933 904 904
Total cold run time: 274114 ms
Total hot run time: 187611 ms
PR approved by at least one committer and no changes requested.