doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[fix](compactiono) fix time series compaction policy in cloud
BUG
In cloud mode, the time series policy does not merge empty versions.
HOW TO FIX
I've added merging of empty versions.
Additionally
In the cloud, single-replica compaction is not needed. I have removed the related code
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TPC-H: Total hot run time: 40091 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d655b396c5f9f2de7220c7cb1b9e0b0ffc2e585c, data reload: false
------ Round 1 ----------------------------------
q1 17670 4591 4474 4474
q2 2571 189 195 189
q3 11449 1308 1207 1207
q4 10495 829 817 817
q5 7643 2718 2649 2649
q6 227 135 135 135
q7 1033 585 577 577
q8 9242 2114 2072 2072
q9 9128 6603 6502 6502
q10 8976 3710 3694 3694
q11 461 241 236 236
q12 480 212 223 212
q13 17765 2962 2951 2951
q14 262 212 209 209
q15 516 461 462 461
q16 525 378 386 378
q17 971 650 684 650
q18 8023 7430 7390 7390
q19 2583 1513 1532 1513
q20 642 309 300 300
q21 5155 3188 3915 3188
q22 350 291 287 287
Total cold run time: 116167 ms
Total hot run time: 40091 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4337 4214 4243 4214
q2 378 272 272 272
q3 3024 2794 2726 2726
q4 1905 1642 1613 1613
q5 5308 5313 5302 5302
q6 211 123 122 122
q7 2251 1894 1938 1894
q8 3217 3412 3393 3393
q9 8515 8511 8539 8511
q10 3896 3652 3710 3652
q11 590 489 494 489
q12 766 593 598 593
q13 16484 2944 2929 2929
q14 290 288 251 251
q15 513 481 477 477
q16 470 411 410 410
q17 1753 1499 1457 1457
q18 7606 7538 7438 7438
q19 6852 1592 1503 1503
q20 2025 1782 1751 1751
q21 5154 4953 4823 4823
q22 584 497 504 497
Total cold run time: 76129 ms
Total hot run time: 54317 ms
TPC-DS: Total hot run time: 185061 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d655b396c5f9f2de7220c7cb1b9e0b0ffc2e585c, data reload: false
query1 917 362 352 352
query2 6458 2554 2401 2401
query3 6657 204 205 204
query4 24644 21128 21326 21128
query5 4125 414 418 414
query6 266 173 170 170
query7 4594 287 295 287
query8 250 194 214 194
query9 8597 2312 2301 2301
query10 433 248 260 248
query11 14690 14117 14106 14106
query12 146 101 95 95
query13 1643 374 368 368
query14 9262 7629 6535 6535
query15 221 171 179 171
query16 7810 278 256 256
query17 1841 570 542 542
query18 1980 275 265 265
query19 196 149 146 146
query20 101 85 85 85
query21 199 127 126 126
query22 5092 4849 4823 4823
query23 34112 33561 33149 33149
query24 10769 2837 2911 2837
query25 648 403 381 381
query26 1701 151 147 147
query27 3107 312 314 312
query28 7760 2014 2009 2009
query29 1047 616 619 616
query30 297 153 156 153
query31 975 755 735 735
query32 88 54 54 54
query33 754 286 251 251
query34 1025 481 476 476
query35 802 697 664 664
query36 1079 916 908 908
query37 140 65 69 65
query38 3175 3018 3019 3018
query39 1623 1532 1544 1532
query40 272 128 126 126
query41 46 43 43 43
query42 105 99 95 95
query43 583 582 565 565
query44 1185 728 753 728
query45 262 228 260 228
query46 1064 765 732 732
query47 1940 1898 1864 1864
query48 376 305 293 293
query49 1165 400 414 400
query50 792 397 381 381
query51 6666 6590 6646 6590
query52 104 92 93 92
query53 359 288 278 278
query54 322 286 253 253
query55 75 72 68 68
query56 235 219 216 216
query57 1220 1145 1126 1126
query58 218 196 213 196
query59 3466 3105 3085 3085
query60 250 231 234 231
query61 88 86 90 86
query62 660 452 459 452
query63 305 275 276 275
query64 9643 7189 7251 7189
query65 3411 3079 3100 3079
query66 1393 338 326 326
query67 15688 15057 14989 14989
query68 9228 544 549 544
query69 541 316 320 316
query70 1368 1082 1089 1082
query71 481 270 270 270
query72 7676 2550 2317 2317
query73 1628 321 318 318
query74 6657 6158 6153 6153
query75 4107 2626 2628 2626
query76 5149 981 947 947
query77 643 260 257 257
query78 10900 10282 10181 10181
query79 8890 517 511 511
query80 1267 425 426 425
query81 501 219 234 219
query82 236 99 93 93
query83 205 167 160 160
query84 259 85 88 85
query85 963 279 261 261
query86 355 322 306 306
query87 3285 3111 3069 3069
query88 4954 2386 2385 2385
query89 525 384 373 373
query90 2054 177 180 177
query91 125 97 95 95
query92 69 46 46 46
query93 6261 514 505 505
query94 1349 188 218 188
query95 401 303 309 303
query96 618 266 266 266
query97 3188 2985 2964 2964
query98 237 219 224 219
query99 1235 893 890 890
Total cold run time: 306472 ms
Total hot run time: 185061 ms
TeamCity be ut coverage result: Function Coverage: 35.66% (8962/25129) Line Coverage: 27.27% (73919/271014) Region Coverage: 26.46% (38180/144293) Branch Coverage: 23.22% (19445/83760) Coverage Report: http://coverage.selectdb-in.cc/coverage/d655b396c5f9f2de7220c7cb1b9e0b0ffc2e585c_d655b396c5f9f2de7220c7cb1b9e0b0ffc2e585c/report/index.html
TPC-H: Total hot run time: 41809 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d655b396c5f9f2de7220c7cb1b9e0b0ffc2e585c, data reload: false
------ Round 1 ----------------------------------
q1 17604 4314 4236 4236
q2 2036 190 191 190
q3 10459 1158 1219 1158
q4 10202 846 838 838
q5 7478 2732 2731 2731
q6 219 138 147 138
q7 1055 626 615 615
q8 9237 2160 2108 2108
q9 10566 6706 6665 6665
q10 9118 3928 3840 3840
q11 468 254 240 240
q12 430 222 226 222
q13 17200 3321 3152 3152
q14 277 237 226 226
q15 515 483 478 478
q16 510 415 393 393
q17 983 627 741 627
q18 8245 7845 7703 7703
q19 2411 1554 1573 1554
q20 644 313 314 313
q21 5414 4103 4170 4103
q22 362 288 279 279
Total cold run time: 115433 ms
Total hot run time: 41809 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4514 4358 4370 4358
q2 363 256 259 256
q3 3222 2898 2873 2873
q4 2018 1723 1778 1723
q5 5313 5491 5532 5491
q6 216 128 130 128
q7 2395 1963 2014 1963
q8 3267 3424 3440 3424
q9 8723 8721 8864 8721
q10 4127 3853 3714 3714
q11 576 473 480 473
q12 814 638 610 610
q13 17027 3104 3231 3104
q14 286 264 291 264
q15 529 470 478 470
q16 490 425 438 425
q17 1830 1573 1495 1495
q18 7799 7398 7518 7398
q19 1665 1556 1478 1478
q20 2001 1770 1757 1757
q21 12331 4942 4990 4942
q22 568 476 484 476
Total cold run time: 80074 ms
Total hot run time: 55543 ms
TPC-DS: Total hot run time: 186730 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d655b396c5f9f2de7220c7cb1b9e0b0ffc2e585c, data reload: false
query1 910 366 351 351
query2 6473 2438 2366 2366
query3 6643 206 206 206
query4 25031 21171 21197 21171
query5 4226 427 444 427
query6 271 186 175 175
query7 4581 292 284 284
query8 248 196 201 196
query9 8574 2303 2295 2295
query10 432 263 257 257
query11 14863 14131 14229 14131
query12 142 92 92 92
query13 1641 367 378 367
query14 9771 7703 8518 7703
query15 250 173 176 173
query16 8073 268 274 268
query17 1723 580 565 565
query18 2107 294 283 283
query19 343 161 156 156
query20 95 88 94 88
query21 195 136 142 136
query22 4975 4807 4808 4807
query23 34304 33291 33431 33291
query24 6902 2965 2927 2927
query25 573 389 398 389
query26 709 165 158 158
query27 1982 323 328 323
query28 4008 2020 2015 2015
query29 855 633 626 626
query30 229 158 159 158
query31 987 764 762 762
query32 87 53 56 53
query33 489 262 255 255
query34 890 488 501 488
query35 803 670 696 670
query36 1057 932 943 932
query37 107 73 70 70
query38 3140 3031 3037 3031
query39 1608 1546 1555 1546
query40 202 138 132 132
query41 45 41 43 41
query42 112 100 97 97
query43 601 560 547 547
query44 1109 744 762 744
query45 278 255 264 255
query46 1059 780 757 757
query47 1937 1867 1866 1866
query48 392 311 304 304
query49 937 404 412 404
query50 798 416 397 397
query51 6697 6689 6659 6659
query52 105 89 93 89
query53 357 286 286 286
query54 265 240 241 240
query55 78 71 75 71
query56 245 227 234 227
query57 1215 1127 1153 1127
query58 217 210 206 206
query59 3423 3072 3166 3072
query60 270 253 259 253
query61 92 91 88 88
query62 615 464 440 440
query63 310 290 288 288
query64 8325 7302 7290 7290
query65 3170 3135 3061 3061
query66 778 340 348 340
query67 15816 14852 14684 14684
query68 9804 557 545 545
query69 615 311 316 311
query70 1331 1099 1073 1073
query71 475 277 275 275
query72 8617 2584 2360 2360
query73 1564 324 324 324
query74 6572 6090 6134 6090
query75 4580 2608 2629 2608
query76 5442 982 1013 982
query77 710 272 270 270
query78 11007 10235 10100 10100
query79 11837 531 531 531
query80 2358 451 448 448
query81 484 230 227 227
query82 232 99 97 97
query83 211 166 167 166
query84 267 86 87 86
query85 954 274 266 266
query86 347 299 302 299
query87 3302 3124 3126 3124
query88 5340 2396 2399 2396
query89 525 405 391 391
query90 2433 187 204 187
query91 126 99 100 99
query92 61 48 49 48
query93 6897 523 506 506
query94 1548 191 186 186
query95 402 307 313 307
query96 632 267 259 259
query97 3149 2966 2978 2966
query98 252 223 217 217
query99 1187 830 885 830
Total cold run time: 303006 ms
Total hot run time: 186730 ms
TeamCity be ut coverage result: Function Coverage: 35.61% (8965/25172) Line Coverage: 27.25% (73980/271481) Region Coverage: 26.47% (38214/144382) Branch Coverage: 23.23% (19462/83790) Coverage Report: http://coverage.selectdb-in.cc/coverage/8c39af32a29f08a57cd5cdc1b0cfbc27c1060f38_8c39af32a29f08a57cd5cdc1b0cfbc27c1060f38/report/index.html
TPC-DS: Total hot run time: 183267 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8c39af32a29f08a57cd5cdc1b0cfbc27c1060f38, data reload: false
query1 918 372 349 349
query2 6166 2363 2438 2363
query3 6650 200 209 200
query4 22998 21670 21698 21670
query5 3875 429 420 420
query6 270 193 182 182
query7 4582 302 296 296
query8 237 185 190 185
query9 8450 2405 2391 2391
query10 405 251 251 251
query11 11881 11209 11525 11209
query12 125 84 87 84
query13 1681 372 377 372
query14 10557 7668 7808 7668
query15 250 168 163 163
query16 8192 258 253 253
query17 1716 556 529 529
query18 2083 282 257 257
query19 306 147 150 147
query20 86 86 81 81
query21 194 131 124 124
query22 5107 4852 4868 4852
query23 33985 33160 33511 33160
query24 10689 2938 2783 2783
query25 598 348 362 348
query26 1356 153 143 143
query27 2872 306 308 306
query28 7453 2041 2044 2041
query29 887 592 582 582
query30 253 143 149 143
query31 929 705 729 705
query32 89 51 54 51
query33 749 245 242 242
query34 1042 464 478 464
query35 783 647 661 647
query36 1069 922 927 922
query37 134 63 63 63
query38 3205 3063 3040 3040
query39 1588 1550 1525 1525
query40 230 124 123 123
query41 38 36 38 36
query42 99 104 97 97
query43 584 557 517 517
query44 1201 728 741 728
query45 265 248 243 243
query46 1071 688 731 688
query47 1988 1892 1934 1892
query48 357 289 286 286
query49 941 375 376 375
query50 754 376 381 376
query51 6696 6629 6621 6621
query52 103 90 91 90
query53 352 270 279 270
query54 299 227 239 227
query55 80 74 77 74
query56 241 217 213 213
query57 1268 1139 1141 1139
query58 227 195 194 194
query59 3440 3070 3149 3070
query60 244 229 231 229
query61 87 86 88 86
query62 654 458 464 458
query63 302 275 275 275
query64 9047 7216 7141 7141
query65 3223 3062 3038 3038
query66 1358 341 335 335
query67 15417 15047 14991 14991
query68 5159 536 538 536
query69 489 305 301 301
query70 1170 1144 1153 1144
query71 408 310 256 256
query72 7039 2529 2385 2385
query73 703 317 310 310
query74 6519 6028 6070 6028
query75 3493 2673 2622 2622
query76 3220 971 959 959
query77 365 268 258 258
query78 10922 10421 10143 10143
query79 3795 514 511 511
query80 2191 415 433 415
query81 529 218 221 218
query82 895 91 91 91
query83 263 174 171 171
query84 273 94 81 81
query85 2174 283 262 262
query86 497 295 290 290
query87 3301 3062 3133 3062
query88 4565 2302 2299 2299
query89 478 370 367 367
query90 1999 180 181 180
query91 123 100 94 94
query92 66 49 46 46
query93 5032 519 502 502
query94 1231 174 181 174
query95 393 302 295 295
query96 591 267 264 264
query97 3106 2938 2998 2938
query98 235 218 216 216
query99 1248 826 862 826
Total cold run time: 287448 ms
Total hot run time: 183267 ms
PR approved by at least one committer and no changes requested.