doris
doris copied to clipboard
apache
[refine](dataque) use lock free queue to impl dataqueue
Proposed changes
Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
TeamCity be ut coverage result: Function Coverage: 35.65% (8969/25162) Line Coverage: 27.29% (74021/271256) Region Coverage: 26.52% (38240/144213) Branch Coverage: 23.32% (19490/83588) Coverage Report: http://coverage.selectdb-in.cc/coverage/497f794e5ebb1c82b10e6efbc267e6b09d639dd6_497f794e5ebb1c82b10e6efbc267e6b09d639dd6/report/index.html
TPC-H: Total hot run time: 41386 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 497f794e5ebb1c82b10e6efbc267e6b09d639dd6, data reload: false
------ Round 1 ----------------------------------
q1 17777 4509 4314 4314
q2 2004 186 196 186
q3 10452 1200 1272 1200
q4 10186 804 806 804
q5 7482 2758 2665 2665
q6 232 131 136 131
q7 979 553 563 553
q8 9246 2188 2138 2138
q9 9542 6820 6766 6766
q10 9323 3983 3887 3887
q11 453 267 245 245
q12 528 218 225 218
q13 17398 3145 3228 3145
q14 263 221 218 218
q15 511 476 449 449
q16 500 423 400 400
q17 980 695 702 695
q18 8317 7823 7804 7804
q19 1718 1578 1578 1578
q20 662 316 312 312
q21 5327 4162 3400 3400
q22 356 303 278 278
Total cold run time: 114236 ms
Total hot run time: 41386 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4525 4376 4417 4376
q2 360 274 274 274
q3 3211 2900 2921 2900
q4 1883 1604 1594 1594
q5 5422 5465 5484 5465
q6 214 124 126 124
q7 1856 1471 1516 1471
q8 3291 3453 3446 3446
q9 8720 8741 8820 8741
q10 4047 3775 3704 3704
q11 593 484 494 484
q12 821 619 626 619
q13 15874 3131 3123 3123
q14 315 279 295 279
q15 521 494 470 470
q16 492 423 431 423
q17 1805 1517 1502 1502
q18 8014 7608 7379 7379
q19 3085 1610 1539 1539
q20 2007 1766 1745 1745
q21 9097 4837 4817 4817
q22 562 489 490 489
Total cold run time: 76715 ms
Total hot run time: 54964 ms
TPC-DS: Total hot run time: 187221 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 497f794e5ebb1c82b10e6efbc267e6b09d639dd6, data reload: false
query1 921 354 346 346
query2 6452 2339 2337 2337
query3 6637 203 206 203
query4 23166 21151 21325 21151
query5 4113 419 420 419
query6 261 179 172 172
query7 4590 297 289 289
query8 240 190 183 183
query9 8576 2455 2433 2433
query10 438 248 249 248
query11 14759 14232 14127 14127
query12 129 91 90 90
query13 1659 390 399 390
query14 9950 8295 8299 8295
query15 260 170 184 170
query16 8132 281 264 264
query17 1793 595 572 572
query18 2110 289 285 285
query19 335 156 160 156
query20 96 88 89 88
query21 200 130 127 127
query22 5118 4951 4893 4893
query23 33798 33067 33121 33067
query24 10475 2927 2929 2927
query25 588 388 387 387
query26 707 164 153 153
query27 2121 325 339 325
query28 5811 2069 2097 2069
query29 881 622 625 622
query30 248 152 153 152
query31 973 736 758 736
query32 97 56 57 56
query33 650 264 250 250
query34 884 474 485 474
query35 799 669 665 665
query36 1043 935 935 935
query37 101 69 69 69
query38 3168 2992 3007 2992
query39 1600 1543 1552 1543
query40 199 130 129 129
query41 43 40 42 40
query42 105 98 101 98
query43 570 545 553 545
query44 1079 738 754 738
query45 269 262 264 262
query46 1093 721 733 721
query47 1958 1909 1899 1899
query48 381 305 301 301
query49 858 479 405 405
query50 780 396 389 389
query51 6640 6681 6529 6529
query52 97 98 89 89
query53 361 286 278 278
query54 308 232 227 227
query55 78 72 72 72
query56 241 221 219 219
query57 1205 1126 1150 1126
query58 216 200 199 199
query59 3402 3335 2977 2977
query60 260 238 252 238
query61 88 90 88 88
query62 612 445 456 445
query63 312 283 279 279
query64 8300 7234 7174 7174
query65 3114 3061 3015 3015
query66 792 343 353 343
query67 15555 15073 15303 15073
query68 9436 561 578 561
query69 563 335 320 320
query70 1209 1101 1133 1101
query71 457 265 270 265
query72 7706 2573 2366 2366
query73 920 327 329 327
query74 6501 6141 6152 6141
query75 4446 2628 2646 2628
query76 5027 1010 939 939
query77 695 261 262 261
query78 10880 10182 10222 10182
query79 11788 525 515 515
query80 1584 480 430 430
query81 489 215 218 215
query82 250 91 96 91
query83 206 163 163 163
query84 266 88 84 84
query85 933 267 263 263
query86 338 311 309 309
query87 3323 3108 3079 3079
query88 5166 2420 2426 2420
query89 521 389 378 378
query90 2246 190 195 190
query91 126 97 97 97
query92 60 50 50 50
query93 6986 530 522 522
query94 1518 184 230 184
query95 402 304 303 303
query96 609 273 267 267
query97 3169 2969 2925 2925
query98 237 215 222 215
query99 1113 872 876 872
Total cold run time: 301965 ms
Total hot run time: 187221 ms
TPC-H: Total hot run time: 39890 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 497f794e5ebb1c82b10e6efbc267e6b09d639dd6, data reload: false
------ Round 1 ----------------------------------
q1 17723 4408 4334 4334
q2 2590 195 202 195
q3 11083 1171 1177 1171
q4 10201 862 751 751
q5 7485 2730 2654 2654
q6 224 135 135 135
q7 1017 597 582 582
q8 9233 2075 2092 2075
q9 9008 6511 6481 6481
q10 8982 3669 3694 3669
q11 467 245 238 238
q12 456 225 228 225
q13 17763 2957 2948 2948
q14 261 230 226 226
q15 505 475 482 475
q16 511 382 374 374
q17 971 637 769 637
q18 7994 7400 7385 7385
q19 6302 1558 1494 1494
q20 654 321 318 318
q21 5072 3276 3237 3237
q22 351 291 286 286
Total cold run time: 118853 ms
Total hot run time: 39890 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4430 4213 4217 4213
q2 384 270 273 270
q3 2964 2817 2714 2714
q4 1852 1643 1576 1576
q5 5284 5285 5288 5285
q6 215 129 126 126
q7 2239 1919 1910 1910
q8 3262 3403 3438 3403
q9 8474 8466 8500 8466
q10 3865 3658 3679 3658
q11 595 478 488 478
q12 762 597 589 589
q13 17322 2986 2971 2971
q14 299 275 260 260
q15 509 480 470 470
q16 462 420 424 420
q17 1820 1510 1514 1510
q18 7663 7565 7314 7314
q19 1710 1565 1594 1565
q20 1970 1758 1795 1758
q21 5172 4916 4874 4874
q22 589 496 496 496
Total cold run time: 71842 ms
Total hot run time: 54326 ms
TeamCity be ut coverage result: Function Coverage: 35.71% (8986/25167) Line Coverage: 27.35% (74219/271350) Region Coverage: 26.59% (38361/144258) Branch Coverage: 23.39% (19555/83616) Coverage Report: http://coverage.selectdb-in.cc/coverage/497f794e5ebb1c82b10e6efbc267e6b09d639dd6_497f794e5ebb1c82b10e6efbc267e6b09d639dd6/report/index.html
TPC-DS: Total hot run time: 185748 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 497f794e5ebb1c82b10e6efbc267e6b09d639dd6, data reload: false
query1 908 366 347 347
query2 6463 2407 2324 2324
query3 6652 209 208 208
query4 23082 21296 21204 21204
query5 4156 424 421 421
query6 275 181 175 175
query7 4595 288 287 287
query8 247 188 196 188
query9 8657 2468 2426 2426
query10 435 256 262 256
query11 14759 14116 14145 14116
query12 137 92 88 88
query13 1669 375 393 375
query14 9028 7718 7504 7504
query15 250 167 175 167
query16 8133 264 256 256
query17 1851 585 543 543
query18 2075 281 278 278
query19 224 150 150 150
query20 93 86 82 82
query21 190 132 127 127
query22 5056 4866 4915 4866
query23 33736 33188 33169 33169
query24 11626 2879 2975 2879
query25 658 368 373 368
query26 1741 159 156 156
query27 3076 322 340 322
query28 7430 2080 2058 2058
query29 1007 605 599 599
query30 277 152 153 152
query31 983 729 741 729
query32 95 54 54 54
query33 748 250 246 246
query34 1117 505 487 487
query35 817 661 679 661
query36 1066 915 900 900
query37 273 68 69 68
query38 2892 2756 2707 2707
query39 1602 1544 1539 1539
query40 276 125 124 124
query41 41 36 35 35
query42 105 96 104 96
query43 575 529 553 529
query44 1239 736 764 736
query45 253 247 253 247
query46 1078 712 744 712
query47 1946 1877 1866 1866
query48 371 300 302 300
query49 1195 426 395 395
query50 785 402 397 397
query51 6829 6721 6694 6694
query52 98 91 93 91
query53 346 281 282 281
query54 324 237 247 237
query55 86 77 73 73
query56 244 226 224 224
query57 1211 1113 1132 1113
query58 233 199 202 199
query59 3531 3100 3166 3100
query60 259 240 239 239
query61 93 91 94 91
query62 672 445 435 435
query63 320 288 283 283
query64 9685 7183 7123 7123
query65 3131 3052 3038 3038
query66 1386 357 349 349
query67 15332 14797 15086 14797
query68 5384 542 555 542
query69 490 320 311 311
query70 1207 1096 1158 1096
query71 408 275 273 273
query72 7386 2565 2356 2356
query73 718 333 341 333
query74 6450 6034 6114 6034
query75 3412 2655 2614 2614
query76 3452 993 979 979
query77 460 273 272 272
query78 10913 10202 10087 10087
query79 4534 544 523 523
query80 1773 443 432 432
query81 519 217 219 217
query82 830 96 101 96
query83 270 172 170 170
query84 269 91 92 91
query85 2119 279 265 265
query86 490 298 285 285
query87 3323 3068 3051 3051
query88 4733 2440 2431 2431
query89 495 403 370 370
query90 2049 189 191 189
query91 128 99 96 96
query92 59 50 48 48
query93 5054 515 519 515
query94 1218 195 196 195
query95 408 309 314 309
query96 608 278 274 274
query97 3146 2951 2934 2934
query98 247 224 213 213
query99 1245 861 867 861
Total cold run time: 293520 ms
Total hot run time: 185748 ms
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TeamCity be ut coverage result: Function Coverage: 35.70% (8983/25159) Line Coverage: 27.36% (74214/271258) Region Coverage: 26.61% (38370/144195) Branch Coverage: 23.42% (19570/83570) Coverage Report: http://coverage.selectdb-in.cc/coverage/2f3a5717d468c6d19a7c5514df4b600be68175b4_2f3a5717d468c6d19a7c5514df4b600be68175b4/report/index.html
TeamCity be ut coverage result: Function Coverage: 35.70% (8983/25161) Line Coverage: 27.36% (74227/271328) Region Coverage: 26.59% (38376/144298) Branch Coverage: 23.41% (19571/83596) Coverage Report: http://coverage.selectdb-in.cc/coverage/d4af77f83080b9733b8703a253b426e1e61dcaec_d4af77f83080b9733b8703a253b426e1e61dcaec/report/index.html
PR approved by at least one committer and no changes requested.
TPC-H: Total hot run time: 41689 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ac72befe48ae21ad140ca8c97eb2a014c7d08788, data reload: false
------ Round 1 ----------------------------------
q1 18355 4399 4290 4290
q2 1872 186 195 186
q3 6026 1303 1397 1303
q4 10098 785 945 785
q5 7496 2672 2648 2648
q6 216 132 137 132
q7 950 553 521 521
q8 9225 2128 2081 2081
q9 7451 7203 7167 7167
q10 8928 3704 3743 3704
q11 468 257 246 246
q12 424 219 217 217
q13 18590 2984 2972 2972
q14 259 214 222 214
q15 513 466 465 465
q16 498 391 375 375
q17 984 677 673 673
q18 7978 7532 7448 7448
q19 1775 1542 1523 1523
q20 686 321 320 320
q21 5140 4128 4191 4128
q22 385 292 291 291
Total cold run time: 108317 ms
Total hot run time: 41689 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4302 4306 4236 4236
q2 371 278 275 275
q3 2991 2767 2745 2745
q4 1908 1616 1600 1600
q5 5306 5301 5341 5301
q6 213 129 125 125
q7 1767 1424 1418 1418
q8 3209 3380 3367 3367
q9 8996 8971 8961 8961
q10 3905 3701 3715 3701
q11 594 520 510 510
q12 766 601 624 601
q13 16991 2966 2974 2966
q14 298 278 275 275
q15 528 482 459 459
q16 461 418 420 418
q17 1780 1488 1456 1456
q18 7557 7584 7367 7367
q19 1732 1584 1615 1584
q20 1965 1769 1757 1757
q21 5060 5074 4926 4926
q22 589 499 504 499
Total cold run time: 71289 ms
Total hot run time: 54547 ms
TeamCity be ut coverage result: Function Coverage: 35.70% (8982/25163) Line Coverage: 27.34% (74196/271419) Region Coverage: 26.58% (38359/144333) Branch Coverage: 23.39% (19561/83612) Coverage Report: http://coverage.selectdb-in.cc/coverage/ac72befe48ae21ad140ca8c97eb2a014c7d08788_ac72befe48ae21ad140ca8c97eb2a014c7d08788/report/index.html
TPC-DS: Total hot run time: 186970 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ac72befe48ae21ad140ca8c97eb2a014c7d08788, data reload: false
query1 910 367 356 356
query2 6676 2299 2349 2299
query3 6652 215 215 215
query4 24059 21327 21370 21327
query5 4152 426 425 425
query6 280 182 186 182
query7 4579 300 295 295
query8 253 186 182 182
query9 8735 2426 2398 2398
query10 436 253 247 247
query11 14764 14248 14158 14158
query12 136 93 89 89
query13 1642 372 367 367
query14 8605 6904 8739 6904
query15 259 171 163 163
query16 8142 268 271 268
query17 1850 553 560 553
query18 2122 289 276 276
query19 286 155 157 155
query20 95 88 90 88
query21 199 133 135 133
query22 5105 4855 4842 4842
query23 34430 33621 33667 33621
query24 10408 2929 2920 2920
query25 585 386 382 382
query26 704 161 157 157
query27 2134 322 327 322
query28 5709 2091 2065 2065
query29 867 632 624 624
query30 283 155 158 155
query31 974 770 746 746
query32 93 59 60 59
query33 664 262 309 262
query34 870 479 488 479
query35 821 711 669 669
query36 1051 924 888 888
query37 101 65 65 65
query38 2890 2807 2793 2793
query39 1651 1554 1559 1554
query40 200 126 120 120
query41 42 38 38 38
query42 105 96 103 96
query43 569 528 542 528
query44 1053 726 735 726
query45 275 257 254 254
query46 1094 714 720 714
query47 1967 1846 1902 1846
query48 372 291 298 291
query49 1041 391 397 391
query50 777 397 385 385
query51 6914 6753 6808 6753
query52 105 91 96 91
query53 349 284 288 284
query54 886 434 459 434
query55 82 82 74 74
query56 240 217 220 217
query57 1238 1127 1155 1127
query58 215 219 201 201
query59 3300 3089 3068 3068
query60 263 236 234 234
query61 93 86 89 86
query62 688 464 474 464
query63 315 293 288 288
query64 8535 7375 7370 7370
query65 3215 3083 3074 3074
query66 820 337 346 337
query67 15500 15031 15208 15031
query68 4615 573 543 543
query69 480 305 300 300
query70 1194 1140 1120 1120
query71 402 267 274 267
query72 7743 2563 2425 2425
query73 705 326 323 323
query74 6598 6240 6265 6240
query75 3467 2696 2610 2610
query76 2907 932 998 932
query77 435 265 274 265
query78 10945 10352 10178 10178
query79 2349 521 527 521
query80 1291 440 424 424
query81 536 218 220 218
query82 661 89 92 89
query83 261 162 164 162
query84 239 90 83 83
query85 1762 312 263 263
query86 489 331 290 290
query87 3307 3158 3106 3106
query88 4069 2428 2409 2409
query89 484 367 367 367
query90 1991 184 185 184
query91 125 96 97 96
query92 62 48 47 47
query93 2372 508 497 497
query94 1196 181 188 181
query95 385 299 297 297
query96 604 279 266 266
query97 3129 2945 3031 2945
query98 228 225 220 220
query99 1247 922 897 897
Total cold run time: 280703 ms
Total hot run time: 186970 ms
TeamCity be ut coverage result: Function Coverage: 35.68% (8980/25165) Line Coverage: 27.35% (74247/271499) Region Coverage: 26.58% (38374/144387) Branch Coverage: 23.40% (19574/83634) Coverage Report: http://coverage.selectdb-in.cc/coverage/e45cf0e55d21e664089956a7e6964103411f08a8_e45cf0e55d21e664089956a7e6964103411f08a8/report/index.html
TPC-H: Total hot run time: 41912 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e45cf0e55d21e664089956a7e6964103411f08a8, data reload: false
------ Round 1 ----------------------------------
q1 17601 4788 4260 4260
q2 2017 193 188 188
q3 10469 1207 1216 1207
q4 10201 802 835 802
q5 7491 2726 2733 2726
q6 220 131 133 131
q7 1043 591 596 591
q8 9232 2229 2115 2115
q9 10650 6770 6719 6719
q10 9421 3946 3847 3847
q11 442 240 254 240
q12 499 227 249 227
q13 17408 3173 3224 3173
q14 255 213 205 205
q15 503 464 465 464
q16 528 393 391 391
q17 985 676 761 676
q18 8373 7686 7739 7686
q19 5933 1586 1516 1516
q20 639 319 330 319
q21 5299 4141 4192 4141
q22 365 288 290 288
Total cold run time: 119574 ms
Total hot run time: 41912 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4538 4392 4450 4392
q2 386 273 275 273
q3 3125 2915 2905 2905
q4 1878 1668 1596 1596
q5 5493 5483 5487 5483
q6 214 123 127 123
q7 2323 1992 1967 1967
q8 3298 3385 3401 3385
q9 8667 8711 8760 8711
q10 3900 3779 3862 3779
q11 599 506 488 488
q12 828 615 620 615
q13 16927 3086 3248 3086
q14 310 276 264 264
q15 517 475 485 475
q16 493 440 429 429
q17 1795 1524 1498 1498
q18 7827 8338 7501 7501
q19 1654 1554 1499 1499
q20 2040 1771 1783 1771
q21 14871 4805 4859 4805
q22 602 508 495 495
Total cold run time: 82285 ms
Total hot run time: 55540 ms
TPC-DS: Total hot run time: 186403 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e45cf0e55d21e664089956a7e6964103411f08a8, data reload: false
query1 922 371 352 352
query2 6428 2443 2361 2361
query3 6644 217 213 213
query4 22807 21288 21243 21243
query5 4077 436 429 429
query6 263 176 171 171
query7 4586 298 290 290
query8 242 199 199 199
query9 8471 2437 2421 2421
query10 434 254 259 254
query11 14989 14183 14227 14183
query12 137 90 90 90
query13 1636 370 364 364
query14 11626 8051 7111 7111
query15 222 175 172 172
query16 7063 272 260 260
query17 1009 552 544 544
query18 1890 275 277 275
query19 206 159 152 152
query20 90 88 82 82
query21 190 138 128 128
query22 5017 4808 4783 4783
query23 33992 33227 33529 33227
query24 5071 2962 2909 2909
query25 470 385 367 367
query26 685 161 157 157
query27 1837 316 330 316
query28 3660 2071 2047 2047
query29 827 624 605 605
query30 224 153 155 153
query31 928 765 728 728
query32 59 54 56 54
query33 408 253 249 249
query34 911 476 493 476
query35 793 703 678 678
query36 1082 916 909 909
query37 105 66 67 66
query38 2830 2806 2757 2757
query39 1627 1680 1568 1568
query40 197 129 133 129
query41 49 45 44 44
query42 107 98 103 98
query43 592 556 591 556
query44 1074 724 732 724
query45 270 258 261 258
query46 1064 734 725 725
query47 1966 1896 1843 1843
query48 372 316 304 304
query49 780 411 418 411
query50 782 405 385 385
query51 6761 6716 6672 6672
query52 102 97 94 94
query53 351 293 295 293
query54 531 431 421 421
query55 75 74 73 73
query56 240 226 230 226
query57 1217 1172 1137 1137
query58 209 202 200 200
query59 3550 3218 3211 3211
query60 290 238 243 238
query61 91 88 89 88
query62 562 458 467 458
query63 306 297 291 291
query64 7689 7391 7387 7387
query65 3151 3117 3091 3091
query66 660 347 339 339
query67 15315 14794 14836 14794
query68 4566 543 540 540
query69 497 307 311 307
query70 1162 1080 1094 1080
query71 455 279 271 271
query72 7403 2579 2370 2370
query73 705 325 328 325
query74 6602 6085 6097 6085
query75 3330 2648 2598 2598
query76 2359 1065 943 943
query77 387 270 274 270
query78 10553 10212 10090 10090
query79 2188 519 517 517
query80 1053 457 445 445
query81 532 223 228 223
query82 698 90 92 90
query83 255 170 169 169
query84 271 88 88 88
query85 1599 272 265 265
query86 522 314 308 308
query87 3281 3167 3108 3108
query88 4263 2426 2405 2405
query89 472 383 406 383
query90 2024 194 192 192
query91 128 101 103 101
query92 65 53 52 52
query93 2730 522 503 503
query94 1263 185 193 185
query95 404 316 326 316
query96 596 268 268 268
query97 3189 2970 3006 2970
query98 234 226 222 222
query99 1224 881 889 881
Total cold run time: 267821 ms
Total hot run time: 186403 ms