doris
doris copied to clipboard
Published
20 hours ago •
apache
apache
[fix](index) should not use light index change for bloom filter index
Proposed changes
Only inverted index support light index change, bitmap and bloomfilter index do not support light index change.
Before this pr, if we add or drop bitmap and bloomfilter index, it only do linked schema change, do not rewrite index data for history data.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TeamCity be ut coverage result: Function Coverage: 35.66% (9014/25279) Line Coverage: 27.30% (74511/272976) Region Coverage: 26.53% (38564/145375) Branch Coverage: 23.37% (19663/84128) Coverage Report: http://coverage.selectdb-in.cc/coverage/50828845400e5bb7e85d8ecd04a8805ddcb4e4d7_50828845400e5bb7e85d8ecd04a8805ddcb4e4d7/report/index.html
TPC-DS: Total hot run time: 170468 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 50828845400e5bb7e85d8ecd04a8805ddcb4e4d7, data reload: false
query1 915 383 373 373
query2 6289 2339 2346 2339
query3 6629 205 205 205
query4 18989 17853 17765 17765
query5 3887 429 412 412
query6 235 177 151 151
query7 4690 322 298 298
query8 254 198 192 192
query9 8798 2483 2435 2435
query10 421 278 276 276
query11 10975 10562 10616 10562
query12 119 89 94 89
query13 1647 351 368 351
query14 9606 7821 7586 7586
query15 245 179 182 179
query16 8339 266 260 260
query17 1766 536 524 524
query18 1929 260 254 254
query19 194 147 153 147
query20 89 78 78 78
query21 189 123 124 123
query22 4324 4004 3868 3868
query23 33610 32871 32972 32871
query24 11118 2817 2753 2753
query25 586 348 344 344
query26 1329 161 149 149
query27 2958 303 301 301
query28 7562 2067 2029 2029
query29 880 623 598 598
query30 255 164 179 164
query31 979 733 742 733
query32 89 49 51 49
query33 758 251 265 251
query34 982 454 474 454
query35 734 586 599 586
query36 1069 895 916 895
query37 135 70 69 69
query38 2910 2771 2747 2747
query39 832 801 791 791
query40 210 123 123 123
query41 44 42 41 41
query42 102 99 98 98
query43 576 524 526 524
query44 1221 740 732 732
query45 175 161 162 161
query46 1076 720 702 702
query47 1860 1759 1773 1759
query48 352 298 287 287
query49 881 375 410 375
query50 746 391 382 382
query51 6955 6772 6826 6772
query52 102 94 93 93
query53 356 287 279 279
query54 903 415 415 415
query55 71 69 73 69
query56 253 240 241 240
query57 1149 1015 1047 1015
query58 226 197 203 197
query59 3258 3235 3120 3120
query60 276 248 250 248
query61 87 85 85 85
query62 676 460 459 459
query63 310 278 289 278
query64 9595 2172 1690 1690
query65 3171 3089 3105 3089
query66 1348 325 318 318
query67 15371 14816 14923 14816
query68 4684 520 525 520
query69 499 268 284 268
query70 1118 1020 1132 1020
query71 458 272 327 272
query72 7407 5405 2698 2698
query73 710 321 317 317
query74 6052 5620 5518 5518
query75 3648 2609 2611 2609
query76 3454 1040 1026 1026
query77 570 264 262 262
query78 10216 9804 9957 9804
query79 2022 526 514 514
query80 823 433 433 433
query81 528 239 253 239
query82 1206 92 95 92
query83 220 169 162 162
query84 231 84 81 81
query85 1489 300 345 300
query86 469 317 285 285
query87 3306 3106 3156 3106
query88 4157 2342 2348 2342
query89 479 370 370 370
query90 1976 189 182 182
query91 121 107 95 95
query92 62 45 47 45
query93 1868 485 490 485
query94 1141 181 180 180
query95 393 297 308 297
query96 600 263 262 262
query97 3173 2995 3044 2995
query98 237 216 211 211
query99 1088 860 891 860
Total cold run time: 273070 ms
Total hot run time: 170468 ms
ClickBench: Total hot run time: 31.07 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 50828845400e5bb7e85d8ecd04a8805ddcb4e4d7, data reload: false
query1 0.04 0.03 0.04
query2 0.09 0.04 0.04
query3 0.24 0.05 0.04
query4 1.67 0.08 0.08
query5 0.50 0.48 0.51
query6 1.13 0.72 0.73
query7 0.02 0.01 0.02
query8 0.05 0.05 0.04
query9 0.54 0.47 0.47
query10 0.52 0.54 0.55
query11 0.16 0.12 0.11
query12 0.15 0.12 0.13
query13 0.59 0.59 0.60
query14 0.76 0.78 0.77
query15 0.83 0.81 0.81
query16 0.36 0.37 0.36
query17 0.98 0.94 1.00
query18 0.22 0.25 0.24
query19 1.79 1.68 1.70
query20 0.01 0.01 0.01
query21 15.54 0.68 0.66
query22 3.65 7.56 2.30
query23 18.25 1.49 1.34
query24 1.55 0.23 0.25
query25 0.15 0.08 0.09
query26 0.25 0.16 0.17
query27 0.08 0.08 0.07
query28 13.58 1.01 0.99
query29 13.79 3.36 3.31
query30 0.25 0.06 0.05
query31 2.84 0.40 0.39
query32 3.25 0.48 0.47
query33 2.92 2.93 2.93
query34 17.25 4.44 4.49
query35 4.48 4.52 4.67
query36 0.65 0.46 0.46
query37 0.18 0.15 0.15
query38 0.15 0.14 0.14
query39 0.04 0.04 0.03
query40 0.16 0.14 0.14
query41 0.09 0.05 0.05
query42 0.05 0.04 0.05
query43 0.04 0.04 0.04
Total cold run time: 109.84 s
Total hot run time: 31.07 s
PR approved by at least one committer and no changes requested.
TPC-H: Total hot run time: 42533 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e3a0f147befa0cb249dc83ca220283f85d112b40, data reload: false
------ Round 1 ----------------------------------
q1 17639 5041 4339 4339
q2 2013 185 206 185
q3 10455 1277 1192 1192
q4 10190 879 820 820
q5 7500 2798 2756 2756
q6 225 133 137 133
q7 969 607 603 603
q8 9211 2174 2172 2172
q9 9518 6851 6766 6766
q10 9336 3999 3899 3899
q11 444 260 232 232
q12 425 230 241 230
q13 17377 3195 3291 3195
q14 266 212 232 212
q15 520 474 486 474
q16 513 403 389 389
q17 1000 807 725 725
q18 8474 7979 7909 7909
q19 6146 1584 1586 1584
q20 642 319 317 317
q21 5257 4104 4170 4104
q22 356 305 297 297
Total cold run time: 118476 ms
Total hot run time: 42533 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4624 4479 4415 4415
q2 386 296 282 282
q3 3160 2977 2872 2872
q4 1933 1669 1648 1648
q5 5511 5510 5527 5510
q6 224 125 126 125
q7 2181 1841 1832 1832
q8 3308 3451 3453 3451
q9 8677 8735 8791 8735
q10 3986 3750 3869 3750
q11 598 503 492 492
q12 836 647 634 634
q13 15999 3160 3129 3129
q14 308 296 285 285
q15 561 491 483 483
q16 512 430 442 430
q17 1829 1560 1524 1524
q18 7923 7672 7603 7603
q19 1712 1623 1555 1555
q20 2009 1807 1815 1807
q21 9077 4853 4709 4709
q22 559 520 475 475
Total cold run time: 75913 ms
Total hot run time: 55746 ms
TeamCity be ut coverage result: Function Coverage: 35.65% (9012/25279) Line Coverage: 27.29% (74514/272999) Region Coverage: 26.53% (38572/145395) Branch Coverage: 23.38% (19672/84144) Coverage Report: http://coverage.selectdb-in.cc/coverage/e3a0f147befa0cb249dc83ca220283f85d112b40_e3a0f147befa0cb249dc83ca220283f85d112b40/report/index.html
TPC-DS: Total hot run time: 173391 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e3a0f147befa0cb249dc83ca220283f85d112b40, data reload: false
query1 924 375 375 375
query2 6445 2528 2349 2349
query3 6639 212 211 211
query4 19294 17473 17457 17457
query5 4083 420 420 420
query6 236 163 151 151
query7 4585 300 293 293
query8 241 185 179 179
query9 8673 2370 2359 2359
query10 456 283 293 283
query11 10444 10199 10134 10134
query12 139 94 89 89
query13 1637 368 370 368
query14 9906 7705 7422 7422
query15 215 175 173 173
query16 7568 261 268 261
query17 1307 538 529 529
query18 1928 283 272 272
query19 197 152 151 151
query20 94 84 85 84
query21 197 128 127 127
query22 4335 4001 4004 4001
query23 33430 33055 33177 33055
query24 5309 2860 2957 2860
query25 471 356 369 356
query26 698 156 157 156
query27 1844 319 325 319
query28 3618 2058 2089 2058
query29 862 606 611 606
query30 245 173 176 173
query31 910 756 766 756
query32 61 53 53 53
query33 495 261 260 260
query34 841 481 474 474
query35 721 606 595 595
query36 1047 905 894 894
query37 104 72 77 72
query38 2902 2824 2761 2761
query39 845 784 797 784
query40 198 123 122 122
query41 47 43 43 43
query42 102 98 94 94
query43 603 545 530 530
query44 1050 716 733 716
query45 174 159 163 159
query46 1053 709 737 709
query47 1873 1772 1750 1750
query48 376 294 290 290
query49 757 378 427 378
query50 771 390 382 382
query51 6899 6606 6689 6606
query52 100 88 88 88
query53 352 291 276 276
query54 535 420 426 420
query55 74 72 74 72
query56 277 240 248 240
query57 1122 1049 1032 1032
query58 236 202 226 202
query59 3494 3203 3093 3093
query60 282 247 250 247
query61 91 90 88 88
query62 548 456 445 445
query63 315 283 281 281
query64 2627 1714 1698 1698
query65 3230 3112 3144 3112
query66 813 326 324 324
query67 15192 15163 14838 14838
query68 4558 537 541 537
query69 437 266 269 266
query70 1105 1131 1106 1106
query71 384 269 258 258
query72 7336 5585 5793 5585
query73 726 322 321 321
query74 5949 5778 5775 5775
query75 3269 2615 2638 2615
query76 2269 982 1026 982
query77 366 261 261 261
query78 10411 9934 9866 9866
query79 2165 518 509 509
query80 879 441 428 428
query81 515 248 244 244
query82 761 95 94 94
query83 259 171 191 171
query84 247 89 85 85
query85 1148 275 275 275
query86 461 305 309 305
query87 3299 3144 3158 3144
query88 3931 2342 2357 2342
query89 481 394 391 391
query90 2066 196 191 191
query91 132 107 106 106
query92 59 51 56 51
query93 2557 513 492 492
query94 1246 192 192 192
query95 414 316 314 314
query96 597 267 266 266
query97 3212 3002 3020 3002
query98 247 218 228 218
query99 1268 967 844 844
Total cold run time: 249891 ms
Total hot run time: 173391 ms
ClickBench: Total hot run time: 30.97 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e3a0f147befa0cb249dc83ca220283f85d112b40, data reload: false
query1 0.04 0.04 0.03
query2 0.08 0.04 0.05
query3 0.23 0.04 0.05
query4 1.69 0.09 0.10
query5 0.55 0.51 0.52
query6 1.12 0.73 0.72
query7 0.02 0.01 0.02
query8 0.05 0.04 0.04
query9 0.53 0.49 0.49
query10 0.55 0.53 0.53
query11 0.16 0.12 0.12
query12 0.15 0.12 0.12
query13 0.59 0.60 0.61
query14 0.76 0.78 0.79
query15 0.84 0.82 0.80
query16 0.35 0.38 0.36
query17 0.94 0.99 1.01
query18 0.22 0.23 0.24
query19 1.74 1.71 1.69
query20 0.01 0.02 0.01
query21 15.44 0.74 0.69
query22 4.51 6.85 2.08
query23 18.28 1.33 1.34
query24 1.86 0.25 0.20
query25 0.15 0.08 0.08
query26 0.27 0.18 0.16
query27 0.08 0.08 0.08
query28 13.26 1.03 1.01
query29 13.33 3.33 3.31
query30 0.24 0.06 0.05
query31 2.87 0.38 0.39
query32 3.27 0.47 0.47
query33 2.93 2.91 2.90
query34 17.03 4.47 4.47
query35 4.59 4.51 4.65
query36 0.66 0.47 0.46
query37 0.18 0.16 0.15
query38 0.15 0.15 0.14
query39 0.04 0.04 0.04
query40 0.16 0.14 0.14
query41 0.09 0.05 0.04
query42 0.06 0.04 0.05
query43 0.04 0.04 0.04
Total cold run time: 110.11 s
Total hot run time: 30.97 s