doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[Fix](flexible partial update) Fix incorrect result when there are insert after delete for same key in one load
Problem
In flexible partial update, if there is an insert after a delete on the same key in one batch, the final result will be always delete that row in table, which is incorrect. The reason is that when doing aggregation for rows with same keys in memtable, we treat delete sign column as an ordinary column. So if there is a row with delete sign in that batch, the final aggregated row will always have a delete sign and any insert on the same key after it will be lost.
Solution
In the above situation, we should keep those rows and don't merge them when doing aggregation in memtable. When flush the memtable to segment, we should first apply the semantic of delete if exists and then insert the row after it.
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TeamCity be ut coverage result: Function Coverage: 37.39% (9659/25830) Line Coverage: 28.70% (80276/279695) Region Coverage: 28.09% (41447/147527) Branch Coverage: 24.70% (21109/85464) Coverage Report: http://coverage.selectdb-in.cc/coverage/8dcd7ae197fa48902b27ffb384aab71195d11193_8dcd7ae197fa48902b27ffb384aab71195d11193/report/index.html
TeamCity be ut coverage result: Function Coverage: 37.39% (9666/25854) Line Coverage: 28.68% (80375/280236) Region Coverage: 28.06% (41451/147730) Branch Coverage: 24.68% (21113/85560) Coverage Report: http://coverage.selectdb-in.cc/coverage/f177418f2be6c8d0161d7df4ecbf5451fd3791c0_f177418f2be6c8d0161d7df4ecbf5451fd3791c0/report/index.html
TPC-H: Total hot run time: 40999 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 79fe26c74dbc3442b2b7b76045ec77054fcb9a10, data reload: false
------ Round 1 ----------------------------------
q1 17607 7641 7313 7313
q2 2020 296 297 296
q3 12122 1077 1199 1077
q4 10576 807 829 807
q5 7771 2904 2832 2832
q6 235 150 149 149
q7 1028 649 614 614
q8 9349 1971 1961 1961
q9 6530 6450 6356 6356
q10 7010 2337 2311 2311
q11 438 248 263 248
q12 427 222 214 214
q13 17783 2989 3005 2989
q14 236 223 214 214
q15 581 526 523 523
q16 654 579 591 579
q17 986 508 536 508
q18 7282 6628 6697 6628
q19 1351 1013 1015 1013
q20 483 204 195 195
q21 3925 3196 3232 3196
q22 1089 1019 976 976
Total cold run time: 109483 ms
Total hot run time: 40999 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7298 7248 7274 7248
q2 325 235 235 235
q3 3022 2971 2954 2954
q4 2105 1846 1823 1823
q5 5795 5762 5737 5737
q6 229 141 149 141
q7 2299 1838 1847 1838
q8 3345 3584 3425 3425
q9 8951 8890 8869 8869
q10 3572 3543 3556 3543
q11 587 514 497 497
q12 826 621 652 621
q13 11172 3210 3135 3135
q14 312 286 283 283
q15 579 546 532 532
q16 705 653 632 632
q17 1861 1592 1607 1592
q18 8275 7796 7444 7444
q19 1703 1412 1541 1412
q20 2098 1877 1896 1877
q21 5577 5413 5509 5413
q22 1146 1061 1057 1057
Total cold run time: 71782 ms
Total hot run time: 60308 ms
TeamCity be ut coverage result: Function Coverage: 37.42% (9681/25871) Line Coverage: 28.70% (80449/280325) Region Coverage: 28.10% (41532/147820) Branch Coverage: 24.68% (21115/85568) Coverage Report: http://coverage.selectdb-in.cc/coverage/79fe26c74dbc3442b2b7b76045ec77054fcb9a10_79fe26c74dbc3442b2b7b76045ec77054fcb9a10/report/index.html
TPC-DS: Total hot run time: 192400 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 79fe26c74dbc3442b2b7b76045ec77054fcb9a10, data reload: false
query1 947 404 412 404
query2 6268 2185 2044 2044
query3 8684 191 200 191
query4 33760 23696 23760 23696
query5 3616 500 468 468
query6 272 168 179 168
query7 4212 305 315 305
query8 290 226 226 226
query9 9293 2679 2676 2676
query10 463 278 266 266
query11 17811 15387 15216 15216
query12 156 106 96 96
query13 1583 472 437 437
query14 9590 7462 7457 7457
query15 246 166 178 166
query16 8042 454 489 454
query17 1634 604 559 559
query18 2088 318 321 318
query19 369 146 145 145
query20 121 107 112 107
query21 213 110 103 103
query22 4946 4362 4529 4362
query23 34854 34245 34311 34245
query24 10977 2867 2848 2848
query25 651 406 400 400
query26 1511 160 158 158
query27 2589 307 292 292
query28 7684 2410 2409 2409
query29 954 426 428 426
query30 262 158 155 155
query31 1023 795 793 793
query32 100 53 55 53
query33 768 303 314 303
query34 908 516 501 501
query35 879 738 736 736
query36 1109 940 932 932
query37 148 84 85 84
query38 4035 3939 3881 3881
query39 1545 1449 1402 1402
query40 255 99 98 98
query41 48 45 44 44
query42 120 95 96 95
query43 556 496 493 493
query44 1217 806 840 806
query45 200 166 159 159
query46 1143 731 713 713
query47 1933 1843 1850 1843
query48 417 358 341 341
query49 981 421 417 417
query50 812 415 414 414
query51 7139 6944 6842 6842
query52 99 86 88 86
query53 253 184 181 181
query54 1401 470 481 470
query55 77 80 79 79
query56 307 272 261 261
query57 1236 1187 1191 1187
query58 256 251 303 251
query59 3281 3221 3028 3028
query60 317 304 287 287
query61 125 119 118 118
query62 854 665 666 665
query63 225 197 190 190
query64 5112 669 601 601
query65 3306 3168 3203 3168
query66 1381 295 326 295
query67 16016 15729 15562 15562
query68 4419 564 562 562
query69 508 295 302 295
query70 1135 1066 1125 1066
query71 369 264 272 264
query72 7516 3927 3928 3927
query73 761 334 344 334
query74 10190 9073 9087 9073
query75 4011 2674 2715 2674
query76 3237 832 878 832
query77 647 330 315 315
query78 10462 9627 9513 9513
query79 1486 590 592 590
query80 2654 432 452 432
query81 596 245 247 245
query82 732 137 138 137
query83 321 142 141 141
query84 277 84 75 75
query85 1698 299 285 285
query86 449 293 299 293
query87 4520 4378 4278 4278
query88 3215 2396 2368 2368
query89 410 288 284 284
query90 2149 185 180 180
query91 138 105 106 105
query92 64 47 48 47
query93 1172 526 538 526
query94 1238 286 295 286
query95 354 255 251 251
query96 608 278 281 278
query97 3248 3130 3122 3122
query98 219 205 190 190
query99 1534 1256 1307 1256
Total cold run time: 304005 ms
Total hot run time: 192400 ms
ClickBench: Total hot run time: 32.6 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 79fe26c74dbc3442b2b7b76045ec77054fcb9a10, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.03 0.04
query3 0.23 0.07 0.07
query4 1.65 0.10 0.10
query5 0.52 0.50 0.50
query6 1.14 0.73 0.72
query7 0.02 0.02 0.02
query8 0.04 0.05 0.03
query9 0.56 0.52 0.51
query10 0.56 0.54 0.56
query11 0.14 0.11 0.11
query12 0.14 0.10 0.10
query13 0.62 0.60 0.59
query14 2.71 2.72 2.85
query15 0.92 0.82 0.85
query16 0.38 0.38 0.38
query17 0.96 1.04 1.00
query18 0.19 0.20 0.19
query19 1.92 1.75 1.89
query20 0.01 0.02 0.01
query21 15.35 0.57 0.55
query22 2.99 2.21 1.95
query23 17.01 0.92 0.79
query24 2.77 2.07 1.00
query25 0.28 0.08 0.12
query26 0.50 0.15 0.13
query27 0.05 0.04 0.04
query28 10.10 1.09 1.08
query29 12.59 3.24 3.19
query30 0.24 0.06 0.05
query31 2.89 0.37 0.38
query32 3.29 0.46 0.47
query33 3.01 3.00 3.02
query34 16.68 4.51 4.45
query35 4.51 4.46 4.50
query36 0.65 0.48 0.48
query37 0.08 0.06 0.06
query38 0.05 0.03 0.04
query39 0.03 0.02 0.03
query40 0.16 0.13 0.12
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.04 0.03 0.03
Total cold run time: 106.2 s
Total hot run time: 32.6 s
TeamCity be ut coverage result: Function Coverage: 37.42% (9681/25873) Line Coverage: 28.63% (80267/280324) Region Coverage: 28.09% (41525/147830) Branch Coverage: 24.67% (21110/85580) Coverage Report: http://coverage.selectdb-in.cc/coverage/02cb8af6bdf15fd2a892f3a48fa8c8b4446761a7_02cb8af6bdf15fd2a892f3a48fa8c8b4446761a7/report/index.html
TPC-H: Total hot run time: 40756 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 02cb8af6bdf15fd2a892f3a48fa8c8b4446761a7, data reload: false
------ Round 1 ----------------------------------
q1 17571 7956 7273 7273
q2 2024 288 271 271
q3 12151 1041 1145 1041
q4 10583 773 706 706
q5 7767 2876 2827 2827
q6 246 156 147 147
q7 1002 633 624 624
q8 9341 1933 2020 1933
q9 6562 6454 6439 6439
q10 6975 2329 2321 2321
q11 449 247 245 245
q12 414 215 216 215
q13 17796 2981 3010 2981
q14 241 209 218 209
q15 568 534 525 525
q16 665 585 588 585
q17 985 571 555 555
q18 7248 6579 6839 6579
q19 1351 964 1057 964
q20 484 201 199 199
q21 3980 3108 3294 3108
q22 1083 1009 1026 1009
Total cold run time: 109486 ms
Total hot run time: 40756 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7263 7218 7239 7218
q2 330 236 233 233
q3 3063 2920 2952 2920
q4 2084 1878 1825 1825
q5 5785 5785 5763 5763
q6 233 141 144 141
q7 2257 1868 1794 1794
q8 3398 3570 3412 3412
q9 8954 8889 8869 8869
q10 3616 3524 3482 3482
q11 580 493 513 493
q12 815 636 658 636
q13 11153 3195 3185 3185
q14 301 284 277 277
q15 595 528 525 525
q16 694 656 649 649
q17 1857 1648 1640 1640
q18 8189 7842 7503 7503
q19 1712 1512 1449 1449
q20 2108 1879 1928 1879
q21 5411 5409 5414 5409
q22 1127 1044 1049 1044
Total cold run time: 71525 ms
Total hot run time: 60346 ms
TPC-DS: Total hot run time: 192006 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 02cb8af6bdf15fd2a892f3a48fa8c8b4446761a7, data reload: false
query1 921 404 388 388
query2 6258 2109 2109 2109
query3 8694 190 208 190
query4 34238 23596 23396 23396
query5 3520 478 474 474
query6 266 165 167 165
query7 4185 299 305 299
query8 287 249 212 212
query9 9532 2654 2645 2645
query10 416 281 284 281
query11 17678 15208 15604 15208
query12 152 97 93 93
query13 1569 459 450 450
query14 9381 7692 7240 7240
query15 242 168 169 168
query16 7296 440 433 433
query17 1667 624 595 595
query18 1393 319 320 319
query19 298 160 154 154
query20 121 113 115 113
query21 206 103 107 103
query22 4746 4496 4326 4326
query23 35012 34207 34207 34207
query24 11103 2856 2916 2856
query25 606 421 410 410
query26 1068 171 168 168
query27 2354 301 300 300
query28 7389 2415 2409 2409
query29 761 447 429 429
query30 259 159 156 156
query31 1058 800 794 794
query32 96 56 56 56
query33 765 299 303 299
query34 896 504 503 503
query35 877 750 728 728
query36 1084 945 951 945
query37 153 92 88 88
query38 3992 3835 3803 3803
query39 1474 1426 1434 1426
query40 206 98 97 97
query41 47 44 46 44
query42 125 102 97 97
query43 533 499 502 499
query44 1217 826 806 806
query45 193 163 168 163
query46 1175 714 726 714
query47 1942 1855 1837 1837
query48 443 364 349 349
query49 928 417 411 411
query50 821 428 423 423
query51 7133 6943 6961 6943
query52 103 91 86 86
query53 256 181 182 181
query54 1241 476 480 476
query55 83 78 75 75
query56 309 280 273 273
query57 1214 1152 1171 1152
query58 242 234 246 234
query59 3392 3107 3079 3079
query60 303 273 266 266
query61 103 99 102 99
query62 878 681 674 674
query63 220 194 190 190
query64 4340 641 610 610
query65 3249 3292 3208 3208
query66 1266 311 306 306
query67 16017 15554 15634 15554
query68 4427 567 535 535
query69 508 306 310 306
query70 1188 1111 1120 1111
query71 368 270 283 270
query72 7185 4027 3992 3992
query73 781 352 354 352
query74 10401 8950 8863 8863
query75 3392 2704 2724 2704
query76 3046 921 865 865
query77 573 298 286 286
query78 10437 9609 9527 9527
query79 3470 613 600 600
query80 2328 455 458 455
query81 587 237 232 232
query82 725 139 137 137
query83 322 129 132 129
query84 281 79 78 78
query85 1122 295 294 294
query86 324 302 306 302
query87 4513 4426 4268 4268
query88 4078 2396 2362 2362
query89 416 287 290 287
query90 2083 187 182 182
query91 151 107 104 104
query92 61 46 48 46
query93 1683 520 533 520
query94 886 306 298 298
query95 354 256 252 252
query96 619 276 273 273
query97 3245 3109 3098 3098
query98 216 217 190 190
query99 1510 1324 1304 1304
Total cold run time: 301272 ms
Total hot run time: 192006 ms
ClickBench: Total hot run time: 33.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 02cb8af6bdf15fd2a892f3a48fa8c8b4446761a7, data reload: false
query1 0.04 0.03 0.03
query2 0.07 0.03 0.03
query3 0.23 0.06 0.06
query4 1.65 0.10 0.10
query5 0.52 0.52 0.50
query6 1.14 0.73 0.73
query7 0.02 0.01 0.02
query8 0.03 0.03 0.03
query9 0.56 0.51 0.49
query10 0.55 0.55 0.56
query11 0.14 0.11 0.11
query12 0.13 0.11 0.11
query13 0.62 0.61 0.60
query14 2.86 2.71 2.88
query15 0.90 0.83 0.83
query16 0.38 0.39 0.38
query17 1.00 1.08 1.02
query18 0.19 0.20 0.20
query19 1.96 1.87 2.00
query20 0.02 0.01 0.00
query21 15.35 0.59 0.60
query22 3.09 1.82 1.83
query23 17.28 1.01 1.04
query24 3.43 1.69 2.09
query25 0.31 0.18 0.24
query26 0.60 0.14 0.14
query27 0.05 0.05 0.03
query28 9.16 1.10 1.08
query29 12.58 3.25 3.16
query30 0.24 0.06 0.06
query31 2.90 0.37 0.37
query32 3.28 0.46 0.45
query33 3.04 3.05 3.08
query34 17.08 4.41 4.50
query35 4.53 4.46 4.52
query36 0.66 0.48 0.48
query37 0.08 0.06 0.05
query38 0.05 0.03 0.04
query39 0.03 0.02 0.02
query40 0.16 0.12 0.13
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.04 0.02 0.03
Total cold run time: 107.05 s
Total hot run time: 33.62 s
TPC-H: Total hot run time: 41508 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 99e53bd3da17aa9c7e310bf4b8d91beb7a81b396, data reload: false
------ Round 1 ----------------------------------
q1 17573 7585 7338 7338
q2 2039 160 158 158
q3 10565 1137 1273 1137
q4 10218 827 856 827
q5 7739 3100 3074 3074
q6 239 155 154 154
q7 1032 632 627 627
q8 9388 1972 1938 1938
q9 7146 6428 6435 6428
q10 7107 2400 2409 2400
q11 457 247 246 246
q12 532 224 236 224
q13 17815 3068 3032 3032
q14 240 216 212 212
q15 575 532 522 522
q16 682 586 583 583
q17 990 574 582 574
q18 7459 6796 6705 6705
q19 1365 995 1098 995
q20 473 187 194 187
q21 4034 3141 3279 3141
q22 1121 1035 1006 1006
Total cold run time: 108789 ms
Total hot run time: 41508 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7240 7256 7262 7256
q2 332 225 232 225
q3 2966 2782 2808 2782
q4 1978 1709 1710 1709
q5 5506 5500 5507 5500
q6 225 142 144 142
q7 2137 1728 1732 1728
q8 3258 3411 3444 3411
q9 8573 8551 8554 8551
q10 3480 3442 3451 3442
q11 571 476 471 471
q12 768 602 588 588
q13 13234 3005 3038 3005
q14 295 253 258 253
q15 551 506 498 498
q16 676 646 643 643
q17 1844 1576 1531 1531
q18 7737 7399 7426 7399
q19 1670 1493 1451 1451
q20 2034 1813 1817 1813
q21 5474 5340 5243 5243
q22 1131 1012 992 992
Total cold run time: 71680 ms
Total hot run time: 58633 ms
TeamCity be ut coverage result: Function Coverage: 37.44% (9708/25932) Line Coverage: 28.67% (80600/281102) Region Coverage: 28.14% (41703/148204) Branch Coverage: 24.71% (21206/85822) Coverage Report: http://coverage.selectdb-in.cc/coverage/99e53bd3da17aa9c7e310bf4b8d91beb7a81b396_99e53bd3da17aa9c7e310bf4b8d91beb7a81b396/report/index.html
TPC-DS: Total hot run time: 192082 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 99e53bd3da17aa9c7e310bf4b8d91beb7a81b396, data reload: false
query1 951 386 393 386
query2 5062 2108 2092 2092
query3 4406 214 220 214
query4 32856 23494 23619 23494
query5 3843 483 468 468
query6 240 175 169 169
query7 3831 299 294 294
query8 286 240 234 234
query9 8374 2781 2743 2743
query10 441 286 274 274
query11 17693 15260 15298 15260
query12 153 99 102 99
query13 1488 445 413 413
query14 10108 7679 7499 7499
query15 292 166 172 166
query16 6715 447 436 436
query17 1528 557 561 557
query18 1838 299 299 299
query19 314 151 145 145
query20 122 107 106 106
query21 213 104 102 102
query22 4634 4392 4266 4266
query23 34799 34155 34033 34033
query24 9899 2799 2835 2799
query25 667 409 403 403
query26 1388 164 159 159
query27 2435 285 282 282
query28 6846 2432 2453 2432
query29 912 435 426 426
query30 327 169 154 154
query31 1056 793 834 793
query32 98 64 63 63
query33 721 297 283 283
query34 940 526 511 511
query35 929 766 803 766
query36 1112 936 950 936
query37 149 98 87 87
query38 4120 3874 3900 3874
query39 1457 1444 1428 1428
query40 286 101 100 100
query41 49 48 44 44
query42 123 103 100 100
query43 541 493 507 493
query44 1210 814 821 814
query45 195 166 164 164
query46 1139 702 702 702
query47 1933 1813 1851 1813
query48 433 320 320 320
query49 1205 417 424 417
query50 816 384 407 384
query51 7030 7028 6983 6983
query52 99 94 88 88
query53 255 180 183 180
query54 917 425 433 425
query55 81 80 75 75
query56 279 268 259 259
query57 1304 1170 1145 1145
query58 237 230 233 230
query59 3290 3127 3108 3108
query60 282 266 261 261
query61 110 106 106 106
query62 882 689 696 689
query63 224 181 187 181
query64 4629 648 599 599
query65 3288 3198 3206 3198
query66 1335 304 301 301
query67 15810 15580 15522 15522
query68 4019 569 558 558
query69 565 275 307 275
query70 1176 1134 1170 1134
query71 362 272 276 272
query72 7090 3977 3936 3936
query73 776 356 359 356
query74 9832 8982 8972 8972
query75 3404 2710 2702 2702
query76 2938 894 969 894
query77 417 288 306 288
query78 10418 9644 9616 9616
query79 3478 609 595 595
query80 1850 447 452 447
query81 590 241 241 241
query82 394 139 145 139
query83 267 137 133 133
query84 266 72 70 70
query85 1146 302 281 281
query86 393 305 300 300
query87 4461 4326 4237 4237
query88 3885 2200 2178 2178
query89 403 321 285 285
query90 1920 185 185 185
query91 145 99 100 99
query92 66 48 48 48
query93 1720 558 559 558
query94 872 294 292 292
query95 337 243 241 241
query96 620 277 276 276
query97 3291 3127 3137 3127
query98 220 198 200 198
query99 1507 1305 1303 1303
Total cold run time: 290053 ms
Total hot run time: 192082 ms
ClickBench: Total hot run time: 32.99 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 99e53bd3da17aa9c7e310bf4b8d91beb7a81b396, data reload: false
query1 0.04 0.04 0.03
query2 0.06 0.03 0.03
query3 0.24 0.06 0.07
query4 1.64 0.11 0.10
query5 0.52 0.50 0.51
query6 1.13 0.74 0.74
query7 0.02 0.02 0.01
query8 0.04 0.03 0.04
query9 0.57 0.50 0.50
query10 0.55 0.54 0.54
query11 0.14 0.11 0.11
query12 0.13 0.11 0.11
query13 0.61 0.59 0.59
query14 2.85 2.70 2.75
query15 0.90 0.83 0.83
query16 0.39 0.38 0.39
query17 1.09 1.04 1.06
query18 0.20 0.20 0.19
query19 1.87 1.90 1.99
query20 0.01 0.01 0.01
query21 15.38 0.60 0.60
query22 2.68 2.10 2.18
query23 17.11 1.10 0.77
query24 3.08 0.97 0.98
query25 0.26 0.13 0.10
query26 0.38 0.13 0.14
query27 0.05 0.04 0.04
query28 10.95 1.09 1.06
query29 12.56 3.23 3.24
query30 0.24 0.06 0.06
query31 2.88 0.38 0.38
query32 3.27 0.47 0.46
query33 3.00 2.98 3.01
query34 17.24 4.46 4.52
query35 4.56 4.57 4.48
query36 0.63 0.48 0.49
query37 0.08 0.06 0.05
query38 0.05 0.04 0.03
query39 0.03 0.02 0.02
query40 0.16 0.12 0.12
query41 0.07 0.02 0.02
query42 0.03 0.02 0.02
query43 0.04 0.03 0.03
Total cold run time: 107.73 s
Total hot run time: 32.99 s
TPC-H: Total hot run time: 41407 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e1f3877e946fc22d7a55d804f2346266653ae2fd, data reload: false
------ Round 1 ----------------------------------
q1 17582 7470 7305 7305
q2 2063 162 153 153
q3 10574 1159 1150 1150
q4 10235 837 896 837
q5 7737 3130 3112 3112
q6 240 159 151 151
q7 1020 618 599 599
q8 9359 1990 2010 1990
q9 6621 6445 6425 6425
q10 7058 2464 2406 2406
q11 443 250 244 244
q12 416 222 226 222
q13 17805 3055 3005 3005
q14 244 209 216 209
q15 559 516 526 516
q16 641 598 581 581
q17 988 566 529 529
q18 7503 6578 6782 6578
q19 1354 948 1018 948
q20 473 190 187 187
q21 4099 3435 3252 3252
q22 1105 1012 1008 1008
Total cold run time: 108119 ms
Total hot run time: 41407 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7328 7247 7234 7234
q2 331 234 224 224
q3 2965 2862 2821 2821
q4 1973 1758 1727 1727
q5 5557 5523 5553 5523
q6 232 144 151 144
q7 2149 1753 1746 1746
q8 3372 3473 3423 3423
q9 8670 8627 8680 8627
q10 3510 3515 3452 3452
q11 585 492 487 487
q12 787 612 573 573
q13 9710 3021 3068 3021
q14 289 263 265 263
q15 554 521 512 512
q16 681 664 629 629
q17 1836 1642 1599 1599
q18 7885 7446 7329 7329
q19 1689 1493 1554 1493
q20 2067 1850 1846 1846
q21 5435 5326 5249 5249
q22 1125 1030 1026 1026
Total cold run time: 68730 ms
Total hot run time: 58948 ms
TPC-DS: Total hot run time: 191875 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e1f3877e946fc22d7a55d804f2346266653ae2fd, data reload: false
query1 971 378 418 378
query2 6517 2136 2100 2100
query3 6781 212 223 212
query4 34366 23597 23660 23597
query5 4422 474 463 463
query6 269 178 180 178
query7 4607 308 295 295
query8 300 241 234 234
query9 9603 2670 2696 2670
query10 494 289 284 284
query11 18035 15355 15132 15132
query12 156 110 101 101
query13 1660 437 409 409
query14 10466 7055 7931 7055
query15 329 181 194 181
query16 7374 477 442 442
query17 1750 583 582 582
query18 1946 326 309 309
query19 379 158 154 154
query20 122 108 108 108
query21 212 108 109 108
query22 4620 4352 4399 4352
query23 35118 34385 34142 34142
query24 11187 2816 2803 2803
query25 662 405 423 405
query26 1174 165 167 165
query27 2430 286 298 286
query28 7747 2437 2415 2415
query29 793 442 434 434
query30 313 163 152 152
query31 1053 809 835 809
query32 100 57 60 57
query33 788 308 294 294
query34 967 531 534 531
query35 920 744 749 744
query36 1131 935 988 935
query37 151 92 88 88
query38 4084 3950 3897 3897
query39 1474 1430 1436 1430
query40 214 99 103 99
query41 51 48 48 48
query42 121 101 100 100
query43 548 496 492 492
query44 1372 830 810 810
query45 203 168 166 166
query46 1162 704 696 696
query47 1916 1835 1793 1793
query48 465 315 328 315
query49 1172 442 430 430
query50 826 400 402 400
query51 7176 6995 6912 6912
query52 107 92 91 91
query53 265 191 181 181
query54 1258 438 429 429
query55 85 79 78 78
query56 291 284 276 276
query57 1325 1176 1133 1133
query58 256 276 248 248
query59 3202 2914 3102 2914
query60 288 271 267 267
query61 113 107 100 100
query62 886 663 672 663
query63 217 193 190 190
query64 4692 646 616 616
query65 3300 3253 3223 3223
query66 1423 327 305 305
query67 16228 15599 15673 15599
query68 4914 556 535 535
query69 444 291 288 288
query70 1219 1144 1139 1139
query71 336 286 283 283
query72 6527 4003 3989 3989
query73 809 360 353 353
query74 10107 8942 9041 8942
query75 3404 2678 2683 2678
query76 2826 1000 928 928
query77 473 297 301 297
query78 10604 9661 9606 9606
query79 1374 608 608 608
query80 2093 452 459 452
query81 595 243 242 242
query82 1224 140 141 140
query83 282 138 133 133
query84 261 72 72 72
query85 1447 295 285 285
query86 394 305 299 299
query87 4442 4321 4351 4321
query88 3412 2204 2180 2180
query89 407 294 290 290
query90 2032 189 191 189
query91 150 99 104 99
query92 71 50 50 50
query93 1084 541 535 535
query94 1165 296 301 296
query95 363 245 247 245
query96 616 276 282 276
query97 3295 3132 3136 3132
query98 229 186 196 186
query99 1639 1321 1302 1302
Total cold run time: 302871 ms
Total hot run time: 191875 ms
ClickBench: Total hot run time: 33.13 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e1f3877e946fc22d7a55d804f2346266653ae2fd, data reload: false
query1 0.04 0.04 0.04
query2 0.06 0.03 0.03
query3 0.22 0.06 0.06
query4 1.65 0.10 0.11
query5 0.51 0.51 0.52
query6 1.13 0.72 0.73
query7 0.02 0.01 0.01
query8 0.04 0.03 0.02
query9 0.56 0.50 0.50
query10 0.55 0.56 0.55
query11 0.14 0.11 0.11
query12 0.14 0.10 0.12
query13 0.61 0.60 0.59
query14 2.72 2.73 2.80
query15 0.90 0.83 0.82
query16 0.38 0.37 0.38
query17 1.06 1.02 1.04
query18 0.23 0.22 0.21
query19 1.96 1.77 2.03
query20 0.01 0.01 0.01
query21 15.37 0.61 0.59
query22 2.56 2.27 2.44
query23 17.04 0.95 1.03
query24 2.86 1.00 0.87
query25 0.25 0.12 0.17
query26 0.35 0.13 0.14
query27 0.04 0.04 0.04
query28 11.01 1.09 1.06
query29 12.52 3.19 3.16
query30 0.24 0.07 0.06
query31 2.88 0.38 0.39
query32 3.25 0.46 0.46
query33 2.95 3.02 3.04
query34 16.75 4.47 4.49
query35 4.49 4.51 4.49
query36 0.68 0.51 0.49
query37 0.09 0.06 0.06
query38 0.05 0.03 0.03
query39 0.03 0.02 0.02
query40 0.17 0.12 0.13
query41 0.08 0.03 0.02
query42 0.03 0.02 0.02
query43 0.04 0.03 0.03
Total cold run time: 106.66 s
Total hot run time: 33.13 s
TeamCity be ut coverage result: Function Coverage: 37.46% (9712/25924) Line Coverage: 28.76% (80654/280481) Region Coverage: 28.17% (41721/148079) Branch Coverage: 24.76% (21209/85674) Coverage Report: http://coverage.selectdb-in.cc/coverage/e1f3877e946fc22d7a55d804f2346266653ae2fd_e1f3877e946fc22d7a55d804f2346266653ae2fd/report/index.html
TPC-H: Total hot run time: 41768 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ae261fd99369ad3a45789fe25ed021eca52ebba7, data reload: false
------ Round 1 ----------------------------------
q1 17858 7772 7409 7409
q2 2035 304 284 284
q3 11810 1086 1186 1086
q4 10571 902 927 902
q5 7782 3127 3091 3091
q6 236 153 148 148
q7 1018 612 609 609
q8 9358 1954 1930 1930
q9 6578 6526 6471 6471
q10 7042 2464 2452 2452
q11 442 238 242 238
q12 417 219 222 219
q13 17773 3006 3055 3006
q14 236 206 217 206
q15 580 516 526 516
q16 638 582 586 582
q17 999 557 631 557
q18 7386 6693 6730 6693
q19 1362 1073 975 975
q20 478 188 188 188
q21 3995 3199 3360 3199
q22 1117 1007 1025 1007
Total cold run time: 109711 ms
Total hot run time: 41768 ms
----- Round 2, with runtime_filter_mode=off -----
q1 7340 7255 7268 7255
q2 342 237 259 237
q3 3041 2981 3019 2981
q4 2085 1822 1801 1801
q5 5806 5811 5809 5809
q6 231 146 146 146
q7 2271 1833 1802 1802
q8 3395 3465 3539 3465
q9 9003 8935 8958 8935
q10 3603 3560 3545 3545
q11 585 502 482 482
q12 860 633 594 594
q13 10445 3193 3204 3193
q14 304 272 271 271
q15 583 527 530 527
q16 709 624 634 624
q17 1876 1628 1641 1628
q18 8431 7833 7522 7522
q19 1730 1515 1426 1426
q20 2165 1890 1853 1853
q21 5555 5373 5447 5373
q22 1151 1062 1100 1062
Total cold run time: 71511 ms
Total hot run time: 60531 ms
TeamCity be ut coverage result: Function Coverage: 37.46% (9710/25920) Line Coverage: 28.75% (80640/280466) Region Coverage: 28.17% (41714/148078) Branch Coverage: 24.75% (21203/85672) Coverage Report: http://coverage.selectdb-in.cc/coverage/ae261fd99369ad3a45789fe25ed021eca52ebba7_ae261fd99369ad3a45789fe25ed021eca52ebba7/report/index.html
TPC-DS: Total hot run time: 192019 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ae261fd99369ad3a45789fe25ed021eca52ebba7, data reload: false
query1 885 393 423 393
query2 6278 2115 2142 2115
query3 8684 194 210 194
query4 34209 23881 23734 23734
query5 3500 483 478 478
query6 294 171 189 171
query7 4194 295 300 295
query8 301 240 237 237
query9 9472 2645 2659 2645
query10 471 277 277 277
query11 17777 15258 15383 15258
query12 159 108 98 98
query13 1574 435 399 399
query14 8882 7065 7429 7065
query15 245 176 175 175
query16 7841 482 457 457
query17 1638 625 593 593
query18 2010 319 314 314
query19 337 155 160 155
query20 121 115 114 114
query21 213 106 109 106
query22 4903 4466 4657 4466
query23 35131 35106 34193 34193
query24 10961 2731 2709 2709
query25 582 411 401 401
query26 900 164 167 164
query27 2312 282 283 282
query28 7581 2433 2409 2409
query29 669 437 424 424
query30 279 154 150 150
query31 1031 813 813 813
query32 99 56 58 56
query33 769 306 300 300
query34 928 511 522 511
query35 884 770 742 742
query36 1115 965 954 954
query37 148 88 88 88
query38 4040 3882 3973 3882
query39 1509 1447 1405 1405
query40 213 99 98 98
query41 49 48 49 48
query42 122 108 98 98
query43 531 498 487 487
query44 1307 820 805 805
query45 202 165 169 165
query46 1137 686 722 686
query47 1984 1840 1872 1840
query48 434 319 335 319
query49 999 441 435 435
query50 818 399 401 399
query51 7024 7167 6927 6927
query52 105 84 86 84
query53 256 179 179 179
query54 1160 443 424 424
query55 80 76 77 76
query56 272 267 257 257
query57 1323 1192 1169 1169
query58 229 254 223 223
query59 3129 3080 2923 2923
query60 298 272 258 258
query61 109 106 99 99
query62 866 655 677 655
query63 225 199 187 187
query64 3905 631 626 626
query65 3266 3157 3196 3157
query66 838 304 309 304
query67 16124 15638 15686 15638
query68 4438 563 563 563
query69 474 278 309 278
query70 1174 1017 1088 1017
query71 355 282 279 279
query72 7017 3906 3983 3906
query73 786 363 364 363
query74 10255 9136 9049 9049
query75 3413 2699 2701 2699
query76 2776 955 1010 955
query77 625 304 292 292
query78 10602 9635 9631 9631
query79 1154 591 603 591
query80 2195 438 458 438
query81 576 240 236 236
query82 736 145 140 140
query83 285 140 135 135
query84 283 71 71 71
query85 1535 311 280 280
query86 380 277 315 277
query87 4499 4330 4409 4330
query88 3369 2203 2167 2167
query89 415 293 285 285
query90 2275 189 190 189
query91 154 101 103 101
query92 65 47 47 47
query93 1069 549 544 544
query94 1158 292 255 255
query95 360 255 251 251
query96 610 282 282 282
query97 3323 3117 3220 3117
query98 233 198 193 193
query99 1605 1270 1296 1270
Total cold run time: 298125 ms
Total hot run time: 192019 ms
ClickBench: Total hot run time: 32.74 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ae261fd99369ad3a45789fe25ed021eca52ebba7, data reload: false
query1 0.03 0.03 0.03
query2 0.07 0.03 0.03
query3 0.24 0.06 0.06
query4 1.65 0.10 0.10
query5 0.54 0.51 0.50
query6 1.13 0.73 0.73
query7 0.02 0.02 0.02
query8 0.04 0.04 0.03
query9 0.56 0.48 0.48
query10 0.56 0.53 0.54
query11 0.14 0.10 0.11
query12 0.14 0.11 0.10
query13 0.61 0.59 0.60
query14 2.72 2.76 2.76
query15 0.90 0.82 0.82
query16 0.39 0.39 0.38
query17 1.03 1.05 1.07
query18 0.19 0.20 0.20
query19 1.93 1.90 1.98
query20 0.01 0.01 0.01
query21 15.36 0.58 0.57
query22 2.33 2.93 2.45
query23 16.94 1.07 0.75
query24 2.96 0.48 2.19
query25 0.29 0.06 0.15
query26 0.57 0.15 0.14
query27 0.04 0.05 0.04
query28 10.27 1.10 1.07
query29 12.56 3.24 3.17
query30 0.25 0.06 0.07
query31 2.87 0.39 0.37
query32 3.28 0.47 0.45
query33 3.05 3.00 3.09
query34 16.83 4.49 4.49
query35 4.53 4.50 4.45
query36 0.66 0.48 0.48
query37 0.08 0.05 0.06
query38 0.04 0.04 0.04
query39 0.04 0.02 0.02
query40 0.16 0.12 0.12
query41 0.08 0.02 0.02
query42 0.03 0.02 0.02
query43 0.04 0.03 0.02
Total cold run time: 106.16 s
Total hot run time: 32.74 s