doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[opt](inverted index) add inlist condition handling to compound
Proposed changes
- Previously, the compound did not support the inlist condition, which could impact performance if an inverted index was created.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TeamCity be ut coverage result: Function Coverage: 35.20% (8919/25339) Line Coverage: 26.98% (73337/271858) Region Coverage: 26.15% (37891/144896) Branch Coverage: 22.97% (19293/84004) Coverage Report: http://coverage.selectdb-in.cc/coverage/99a0b461bb4c06e064ba6da7aadba8052a0ee673_99a0b461bb4c06e064ba6da7aadba8052a0ee673/report/index.html
TeamCity be ut coverage result: Function Coverage: 35.20% (8919/25340) Line Coverage: 26.97% (73337/271873) Region Coverage: 26.15% (37894/144905) Branch Coverage: 22.96% (19292/84008) Coverage Report: http://coverage.selectdb-in.cc/coverage/5ef208533dee688ba393bf4c7c338abba92b146f_5ef208533dee688ba393bf4c7c338abba92b146f/report/index.html
TPC-H: Total hot run time: 40275 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5ef208533dee688ba393bf4c7c338abba92b146f, data reload: false
------ Round 1 ----------------------------------
q1 17604 4332 4245 4245
q2 2023 195 191 191
q3 10465 1175 1178 1175
q4 10180 825 762 762
q5 7551 2690 2694 2690
q6 218 133 132 132
q7 1012 604 575 575
q8 9231 2086 2054 2054
q9 8973 6614 6592 6592
q10 8785 3756 3748 3748
q11 449 234 248 234
q12 477 227 216 216
q13 17776 2962 3015 2962
q14 276 231 233 231
q15 540 479 479 479
q16 533 387 383 383
q17 967 736 707 707
q18 8139 7543 7630 7543
q19 5462 1523 1503 1503
q20 646 319 299 299
q21 5050 3287 3415 3287
q22 344 267 276 267
Total cold run time: 116701 ms
Total hot run time: 40275 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4319 4228 4204 4204
q2 375 271 275 271
q3 2989 2764 2786 2764
q4 1857 1575 1583 1575
q5 5298 5330 5351 5330
q6 207 122 124 122
q7 2249 1841 1833 1833
q8 3208 3366 3336 3336
q9 8648 8602 9038 8602
q10 4194 3973 4030 3973
q11 636 503 512 503
q12 826 730 681 681
q13 16550 3202 3182 3182
q14 316 287 283 283
q15 511 489 504 489
q16 509 451 436 436
q17 1831 1523 1509 1509
q18 8002 8240 7602 7602
q19 1682 1622 1605 1605
q20 2048 1885 1829 1829
q21 5168 5028 5001 5001
q22 533 486 489 486
Total cold run time: 71956 ms
Total hot run time: 55616 ms
TPC-DS: Total hot run time: 188263 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5ef208533dee688ba393bf4c7c338abba92b146f, data reload: false
query1 892 375 362 362
query2 6172 2635 2354 2354
query3 6648 208 201 201
query4 24643 21629 21435 21435
query5 3918 408 408 408
query6 273 194 198 194
query7 4586 291 287 287
query8 238 183 186 183
query9 8487 2280 2262 2262
query10 411 243 253 243
query11 14808 14225 14180 14180
query12 139 90 83 83
query13 1638 365 364 364
query14 10578 7593 8431 7593
query15 261 190 188 188
query16 8246 277 265 265
query17 1951 584 569 569
query18 2103 287 290 287
query19 339 154 159 154
query20 92 89 86 86
query21 203 131 129 129
query22 4993 4892 4917 4892
query23 33988 33207 33542 33207
query24 10791 3024 3091 3024
query25 608 417 419 417
query26 705 175 170 170
query27 2239 377 370 370
query28 5557 2067 2010 2010
query29 926 612 619 612
query30 235 153 162 153
query31 1017 793 772 772
query32 85 54 58 54
query33 699 252 253 252
query34 1045 510 506 506
query35 839 720 748 720
query36 1095 951 918 918
query37 105 66 70 66
query38 3490 3349 3391 3349
query39 1652 1579 1569 1569
query40 174 128 127 127
query41 42 37 36 36
query42 109 101 97 97
query43 581 539 543 539
query44 1135 749 751 749
query45 296 262 277 262
query46 1116 750 740 740
query47 2032 1996 1993 1993
query48 379 307 325 307
query49 823 401 394 394
query50 768 397 420 397
query51 6877 6775 6873 6775
query52 106 97 93 93
query53 340 270 277 270
query54 309 228 224 224
query55 75 75 71 71
query56 231 219 223 219
query57 1217 1123 1135 1123
query58 226 193 196 193
query59 3321 3293 3166 3166
query60 263 229 226 226
query61 88 88 84 84
query62 601 431 443 431
query63 309 271 280 271
query64 8078 7196 7155 7155
query65 3099 3022 3039 3022
query66 745 337 331 331
query67 15451 15082 14871 14871
query68 8829 554 536 536
query69 542 302 308 302
query70 1243 1144 1091 1091
query71 486 275 272 272
query72 7162 2599 2437 2437
query73 981 320 317 317
query74 6828 6433 6432 6432
query75 3954 2683 2685 2683
query76 4833 992 936 936
query77 615 260 261 260
query78 11003 10227 10216 10216
query79 7306 514 519 514
query80 1347 444 430 430
query81 480 220 219 219
query82 242 89 98 89
query83 195 172 167 167
query84 262 86 82 82
query85 817 264 260 260
query86 344 312 310 310
query87 3476 3329 3269 3269
query88 4737 2400 2393 2393
query89 494 376 369 369
query90 1906 188 188 188
query91 138 148 103 103
query92 56 50 49 49
query93 5827 504 503 503
query94 1090 182 180 180
query95 393 305 303 303
query96 611 269 269 269
query97 3121 2978 3001 2978
query98 260 218 217 217
query99 1164 847 835 835
Total cold run time: 296228 ms
Total hot run time: 188263 ms
TPC-H: Total hot run time: 39885 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d0fda9252625e7cd9a581df07bd138843a2a0928, data reload: false
------ Round 1 ----------------------------------
q1 12335 4277 4263 4263
q2 1885 189 190 189
q3 10190 1129 1219 1129
q4 9713 837 811 811
q5 7542 2731 2669 2669
q6 212 128 130 128
q7 1013 599 587 587
q8 9220 2053 2045 2045
q9 8903 6616 6605 6605
q10 6885 3706 3775 3706
q11 355 235 230 230
q12 383 221 215 215
q13 16438 2931 2956 2931
q14 277 231 229 229
q15 514 489 473 473
q16 499 380 378 378
q17 942 715 637 637
q18 8032 7518 7384 7384
q19 1553 1542 1480 1480
q20 499 321 310 310
q21 5019 3209 3955 3209
q22 323 277 284 277
Total cold run time: 102732 ms
Total hot run time: 39885 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4242 4201 4198 4198
q2 365 277 269 269
q3 3030 2770 2719 2719
q4 1826 1647 1592 1592
q5 5297 5327 5327 5327
q6 210 121 123 121
q7 2264 1893 1899 1893
q8 3213 3400 3370 3370
q9 8579 8583 8545 8545
q10 4036 3713 3712 3712
q11 573 499 500 499
q12 774 587 580 580
q13 16509 2992 2911 2911
q14 306 297 281 281
q15 512 487 470 470
q16 470 430 435 430
q17 1738 1470 1475 1470
q18 7700 7628 7395 7395
q19 3004 1556 1529 1529
q20 1995 1775 1840 1775
q21 5002 4770 4957 4770
q22 591 495 486 486
Total cold run time: 72236 ms
Total hot run time: 54342 ms
TeamCity be ut coverage result: Function Coverage: 35.18% (8919/25349) Line Coverage: 26.97% (73355/271971) Region Coverage: 26.15% (37899/144954) Branch Coverage: 22.96% (19294/84026) Coverage Report: http://coverage.selectdb-in.cc/coverage/d0fda9252625e7cd9a581df07bd138843a2a0928_d0fda9252625e7cd9a581df07bd138843a2a0928/report/index.html
TPC-DS: Total hot run time: 187125 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d0fda9252625e7cd9a581df07bd138843a2a0928, data reload: false
query1 916 375 348 348
query2 6455 2468 2392 2392
query3 6659 204 206 204
query4 23895 21838 21875 21838
query5 4254 432 459 432
query6 561 204 199 199
query7 5896 302 317 302
query8 463 209 188 188
query9 10708 2322 2321 2321
query10 1361 441 282 282
query11 14863 14204 14267 14204
query12 133 86 86 86
query13 1630 362 356 356
query14 9721 7482 6645 6645
query15 263 174 177 174
query16 8235 255 252 252
query17 1983 575 537 537
query18 2110 281 266 266
query19 330 147 144 144
query20 92 85 83 83
query21 192 127 125 125
query22 5072 4837 4908 4837
query23 34360 33403 33241 33241
query24 10764 3003 2901 2901
query25 607 376 369 369
query26 1162 158 149 149
query27 2325 316 322 316
query28 7208 1976 1963 1963
query29 843 603 585 585
query30 294 151 157 151
query31 979 756 700 700
query32 90 53 54 53
query33 745 246 241 241
query34 1057 460 466 460
query35 826 671 667 667
query36 1098 896 879 879
query37 140 70 64 64
query38 3134 3017 2996 2996
query39 1597 1536 1540 1536
query40 198 125 126 125
query41 43 43 38 38
query42 104 94 94 94
query43 582 541 538 538
query44 1228 734 743 734
query45 281 260 272 260
query46 1081 719 719 719
query47 1956 1864 1867 1864
query48 369 293 295 293
query49 1012 417 398 398
query50 778 387 384 384
query51 7012 6763 6718 6718
query52 101 89 92 89
query53 343 276 277 276
query54 312 228 237 228
query55 78 71 72 71
query56 235 241 227 227
query57 1185 1122 1102 1102
query58 241 199 201 199
query59 3495 3136 3183 3136
query60 255 232 235 232
query61 92 104 111 104
query62 682 455 446 446
query63 304 274 274 274
query64 8600 7180 7108 7108
query65 3103 3029 3012 3012
query66 838 344 345 344
query67 15364 15041 14991 14991
query68 5399 543 530 530
query69 490 304 303 303
query70 1127 1155 1024 1024
query71 437 270 273 270
query72 7743 2759 2584 2584
query73 707 320 333 320
query74 6534 6135 6145 6135
query75 3365 2645 2710 2645
query76 3468 961 938 938
query77 431 277 268 268
query78 10956 10290 10184 10184
query79 7188 510 513 510
query80 1855 460 451 451
query81 546 233 223 223
query82 936 95 95 95
query83 264 172 172 172
query84 267 91 92 91
query85 2009 330 319 319
query86 488 314 327 314
query87 3326 3118 3098 3098
query88 5060 2398 2388 2388
query89 486 377 379 377
query90 2058 188 180 180
query91 118 97 95 95
query92 60 49 47 47
query93 5431 505 495 495
query94 1172 182 194 182
query95 1102 1100 1094 1094
query96 608 263 266 263
query97 3146 2971 2959 2959
query98 236 226 221 221
query99 1229 898 860 860
Total cold run time: 301135 ms
Total hot run time: 187125 ms
TeamCity be ut coverage result: Function Coverage: 35.49% (8923/25144) Line Coverage: 27.13% (73446/270757) Region Coverage: 26.30% (37931/144223) Branch Coverage: 23.10% (19324/83668) Coverage Report: http://coverage.selectdb-in.cc/coverage/87324f235c8d62cf76832beb50704befd36fecd8_87324f235c8d62cf76832beb50704befd36fecd8/report/index.html
TPC-H: Total hot run time: 41644 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 87324f235c8d62cf76832beb50704befd36fecd8, data reload: false
------ Round 1 ----------------------------------
q1 17603 4379 4217 4217
q2 2019 197 190 190
q3 10454 1225 1184 1184
q4 10214 888 855 855
q5 7540 2768 2781 2768
q6 225 135 131 131
q7 1072 625 625 625
q8 9231 2166 2132 2132
q9 9182 6782 6768 6768
q10 9036 3939 3891 3891
q11 445 262 235 235
q12 454 232 238 232
q13 17701 3263 3195 3195
q14 302 253 242 242
q15 531 465 484 465
q16 514 414 407 407
q17 967 661 688 661
q18 8461 7773 7824 7773
q19 4292 1561 1562 1561
q20 647 328 334 328
q21 5307 4036 3503 3503
q22 352 283 281 281
Total cold run time: 116549 ms
Total hot run time: 41644 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4492 4396 4379 4379
q2 395 276 270 270
q3 3218 2915 2911 2911
q4 2025 1748 1598 1598
q5 5429 5520 5541 5520
q6 215 124 133 124
q7 2358 1973 1999 1973
q8 3272 3434 3452 3434
q9 8927 8886 8938 8886
q10 4116 3848 3805 3805
q11 586 481 507 481
q12 796 634 627 627
q13 16981 3101 3207 3101
q14 314 300 319 300
q15 534 483 497 483
q16 489 441 440 440
q17 1806 1516 1534 1516
q18 7816 7633 7328 7328
q19 1634 1543 1536 1536
q20 2001 1831 1751 1751
q21 8875 4751 4702 4702
q22 555 482 481 481
Total cold run time: 76834 ms
Total hot run time: 55646 ms
TPC-DS: Total hot run time: 186577 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 87324f235c8d62cf76832beb50704befd36fecd8, data reload: false
query1 911 373 354 354
query2 7175 2403 2358 2358
query3 6632 213 213 213
query4 24007 21233 21345 21233
query5 4121 435 435 435
query6 267 198 192 192
query7 4585 290 299 290
query8 251 206 194 194
query9 8447 2327 2313 2313
query10 435 251 256 251
query11 14709 14329 14139 14139
query12 139 88 85 85
query13 1641 385 373 373
query14 10503 7626 7558 7558
query15 261 180 173 173
query16 8065 247 255 247
query17 1747 550 537 537
query18 2097 281 274 274
query19 272 153 150 150
query20 94 92 90 90
query21 192 127 123 123
query22 5034 4846 4819 4819
query23 33934 33209 33205 33205
query24 10318 2946 2919 2919
query25 574 370 376 370
query26 700 156 150 150
query27 2119 318 328 318
query28 5847 2019 1995 1995
query29 841 600 602 600
query30 264 153 149 149
query31 999 732 720 720
query32 90 52 54 52
query33 629 248 243 243
query34 888 470 477 470
query35 806 696 658 658
query36 1087 952 924 924
query37 110 65 71 65
query38 3237 3057 3038 3038
query39 1614 1527 1538 1527
query40 197 131 124 124
query41 43 39 39 39
query42 106 96 95 95
query43 557 551 557 551
query44 1082 729 732 729
query45 281 269 254 254
query46 1076 740 712 712
query47 1948 1830 1881 1830
query48 384 301 298 298
query49 824 400 403 400
query50 779 393 381 381
query51 6871 6744 6677 6677
query52 100 97 88 88
query53 355 282 288 282
query54 306 242 247 242
query55 81 78 74 74
query56 244 224 226 224
query57 1205 1134 1137 1134
query58 228 209 199 199
query59 3343 3193 3214 3193
query60 265 240 238 238
query61 91 87 99 87
query62 634 450 432 432
query63 314 299 289 289
query64 8161 7154 7181 7154
query65 3078 3054 3061 3054
query66 767 343 338 338
query67 15398 14835 15164 14835
query68 5467 529 541 529
query69 537 305 296 296
query70 1215 1154 1123 1123
query71 419 277 275 275
query72 7847 2641 2398 2398
query73 719 332 327 327
query74 6573 6137 6156 6137
query75 3452 2705 2679 2679
query76 3731 1002 984 984
query77 610 268 270 268
query78 11119 10360 10278 10278
query79 6027 519 522 519
query80 1109 480 448 448
query81 491 222 220 220
query82 1399 104 96 96
query83 207 184 167 167
query84 270 83 84 83
query85 1213 269 258 258
query86 446 322 318 318
query87 3285 3123 3097 3097
query88 4855 2359 2354 2354
query89 493 380 376 376
query90 2041 187 199 187
query91 133 98 99 98
query92 59 50 51 50
query93 5581 517 505 505
query94 1208 183 183 183
query95 400 307 302 302
query96 593 271 267 267
query97 3157 2928 2925 2925
query98 232 224 217 217
query99 1234 864 856 856
Total cold run time: 290482 ms
Total hot run time: 186577 ms
TeamCity be ut coverage result: Function Coverage: 35.59% (8933/25098) Line Coverage: 27.21% (73658/270684) Region Coverage: 26.40% (38053/144147) Branch Coverage: 23.17% (19390/83684) Coverage Report: http://coverage.selectdb-in.cc/coverage/5f6f93f43ff84c00d65fbef2966e758acca0b887_5f6f93f43ff84c00d65fbef2966e758acca0b887/report/index.html
TPC-H: Total hot run time: 41387 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5f6f93f43ff84c00d65fbef2966e758acca0b887, data reload: false
------ Round 1 ----------------------------------
q1 17687 4394 4301 4301
q2 2025 190 202 190
q3 10525 1199 1230 1199
q4 10206 805 760 760
q5 7538 2833 2764 2764
q6 219 134 132 132
q7 1034 616 631 616
q8 9216 2129 2086 2086
q9 9164 6845 6737 6737
q10 9191 4020 3948 3948
q11 475 249 241 241
q12 491 217 232 217
q13 17549 3222 3084 3084
q14 257 225 221 221
q15 522 492 478 478
q16 496 395 399 395
q17 977 678 736 678
q18 8561 7757 7730 7730
q19 5264 1572 1562 1562
q20 657 319 316 316
q21 5195 3447 4184 3447
q22 357 285 285 285
Total cold run time: 117606 ms
Total hot run time: 41387 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4561 4435 4396 4396
q2 389 272 258 258
q3 3247 2926 2845 2845
q4 1917 1663 1606 1606
q5 5540 5521 5561 5521
q6 213 127 122 122
q7 2369 1974 1973 1973
q8 3245 3416 3435 3416
q9 8872 8900 8910 8900
q10 3941 3808 3824 3808
q11 588 485 515 485
q12 841 609 642 609
q13 16987 3048 3108 3048
q14 315 285 295 285
q15 512 469 493 469
q16 493 420 415 415
q17 1765 1492 1443 1443
q18 7686 7680 7472 7472
q19 1638 1512 1558 1512
q20 1996 1772 1737 1737
q21 13775 4747 4874 4747
q22 548 517 512 512
Total cold run time: 81438 ms
Total hot run time: 55579 ms
TPC-DS: Total hot run time: 185556 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5f6f93f43ff84c00d65fbef2966e758acca0b887, data reload: false
query1 911 361 349 349
query2 6437 2370 2345 2345
query3 6668 216 220 216
query4 25878 21161 21197 21161
query5 4097 411 416 411
query6 274 168 168 168
query7 4590 286 283 283
query8 241 183 195 183
query9 8579 2337 2318 2318
query10 427 251 250 250
query11 14784 14129 14200 14129
query12 134 88 87 87
query13 1646 374 357 357
query14 8659 8361 6716 6716
query15 206 167 172 167
query16 7125 256 256 256
query17 1035 544 541 541
query18 1911 272 266 266
query19 198 144 151 144
query20 92 88 84 84
query21 194 122 125 122
query22 5042 4938 4863 4863
query23 33806 33267 33224 33224
query24 5105 2896 3001 2896
query25 480 386 372 372
query26 684 157 150 150
query27 1838 314 326 314
query28 3491 2003 1990 1990
query29 801 619 613 613
query30 242 160 159 159
query31 925 738 718 718
query32 60 51 52 51
query33 413 249 246 246
query34 863 474 485 474
query35 752 666 667 666
query36 1032 910 935 910
query37 104 64 66 64
query38 3101 3003 3051 3003
query39 1622 1548 1523 1523
query40 194 124 125 124
query41 43 37 36 36
query42 104 98 98 98
query43 562 552 531 531
query44 1053 731 747 731
query45 270 261 257 257
query46 1053 748 749 748
query47 1963 1880 1862 1862
query48 368 297 302 297
query49 741 410 433 410
query50 773 384 384 384
query51 6950 6816 6701 6701
query52 104 91 91 91
query53 354 284 288 284
query54 262 229 237 229
query55 80 76 79 76
query56 235 219 218 218
query57 1197 1163 1121 1121
query58 216 193 196 193
query59 3217 3179 3004 3004
query60 254 229 231 229
query61 89 87 87 87
query62 551 453 446 446
query63 304 276 289 276
query64 7486 7239 7137 7137
query65 3109 3037 3060 3037
query66 666 335 336 335
query67 15536 15029 14941 14941
query68 5284 546 541 541
query69 492 295 298 295
query70 1160 1065 1065 1065
query71 454 265 261 261
query72 7613 2738 2534 2534
query73 699 326 331 326
query74 6450 6082 6248 6082
query75 3368 2676 2674 2674
query76 2798 966 1004 966
query77 420 272 273 272
query78 11013 10406 10225 10225
query79 7773 532 539 532
query80 2266 464 467 464
query81 537 234 223 223
query82 1273 97 97 97
query83 278 174 172 172
query84 270 91 89 89
query85 1690 318 316 316
query86 487 315 306 306
query87 3331 3098 3161 3098
query88 4905 2366 2368 2366
query89 508 395 395 395
query90 2035 190 189 189
query91 135 106 111 106
query92 67 50 51 50
query93 6684 516 500 500
query94 1267 181 185 181
query95 397 303 301 301
query96 613 270 275 270
query97 3126 2922 2955 2922
query98 237 229 213 213
query99 1199 871 853 853
Total cold run time: 281010 ms
Total hot run time: 185556 ms