doris
doris copied to clipboard
apache
[refactor](move-memtable) remove IndexStream in LoadStream
Proposed changes
tablet_id is the unique identifier of a TabletStream.
IndexStream is unnecessary in LoadStream, remove it to simplify code.
Also refactor the close logic of LoadStream, and add some timer logs.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR
Since 2024-03-18, the Document has been moved to doris-website. See Doris Document.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
TeamCity be ut coverage result: Function Coverage: 35.69% (9016/25259) Line Coverage: 27.33% (74534/272672) Region Coverage: 26.55% (38522/145118) Branch Coverage: 23.40% (19659/84014) Coverage Report: http://coverage.selectdb-in.cc/coverage/abea6021a783a69de1bc69ca125aa8eff596c260_abea6021a783a69de1bc69ca125aa8eff596c260/report/index.html
PR approved by at least one committer and no changes requested.
TPC-DS: Total hot run time: 182225 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit abea6021a783a69de1bc69ca125aa8eff596c260, data reload: false
query1 919 394 375 375
query2 6478 2398 2262 2262
query3 6651 220 222 220
query4 22842 21206 21197 21197
query5 4133 413 416 413
query6 269 174 197 174
query7 4598 286 286 286
query8 239 187 192 187
query9 8639 2450 2380 2380
query10 430 256 257 256
query11 14859 14062 14097 14062
query12 127 90 87 87
query13 1643 363 356 356
query14 9730 7679 8158 7679
query15 235 168 181 168
query16 7891 261 252 252
query17 1893 582 535 535
query18 1990 277 278 277
query19 200 147 148 147
query20 94 89 85 85
query21 191 132 127 127
query22 5129 4959 4930 4930
query23 34350 33638 33587 33587
query24 11152 2931 2879 2879
query25 614 373 369 369
query26 1192 160 154 154
query27 3026 321 312 312
query28 7244 2056 2022 2022
query29 862 613 600 600
query30 309 176 176 176
query31 984 748 752 748
query32 92 52 52 52
query33 747 254 254 254
query34 1040 478 475 475
query35 820 705 710 705
query36 1097 909 941 909
query37 148 71 71 71
query38 2938 2848 2781 2781
query39 1624 1594 1555 1555
query40 204 132 136 132
query41 48 47 46 46
query42 114 97 102 97
query43 553 535 549 535
query44 1240 730 749 730
query45 277 269 268 268
query46 1074 739 712 712
query47 1980 1891 1892 1891
query48 381 310 302 302
query49 1097 439 422 422
query50 786 384 390 384
query51 6871 6777 6690 6690
query52 106 97 95 95
query53 356 291 278 278
query54 932 432 429 429
query55 74 76 74 74
query56 247 228 233 228
query57 1274 1172 1146 1146
query58 248 204 207 204
query59 3586 3116 3150 3116
query60 270 242 246 242
query61 112 104 106 104
query62 664 484 468 468
query63 314 287 284 284
query64 10047 2297 1781 1781
query65 3224 3100 3096 3096
query66 1407 357 347 347
query67 15518 15242 15061 15061
query68 4634 530 536 530
query69 497 308 317 308
query70 1177 1105 1168 1105
query71 416 271 272 271
query72 7591 2709 2486 2486
query73 705 323 321 321
query74 6796 6271 6343 6271
query75 3393 2668 2601 2601
query76 2399 1011 1096 1011
query77 447 277 269 269
query78 10699 10363 10102 10102
query79 4279 520 525 520
query80 1884 435 433 433
query81 548 246 246 246
query82 1008 98 104 98
query83 267 170 172 170
query84 266 85 87 85
query85 2136 264 257 257
query86 503 299 307 299
query87 3325 3183 3130 3130
query88 4812 2361 2396 2361
query89 487 379 409 379
query90 2003 191 185 185
query91 123 105 98 98
query92 67 49 48 48
query93 5175 526 497 497
query94 1240 180 184 180
query95 397 302 306 302
query96 618 273 269 269
query97 3255 3030 3057 3030
query98 246 220 212 212
query99 1273 885 908 885
Total cold run time: 293089 ms
Total hot run time: 182225 ms
ClickBench: Total hot run time: 30.58 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit abea6021a783a69de1bc69ca125aa8eff596c260, data reload: false
query1 0.04 0.03 0.03
query2 0.09 0.04 0.04
query3 0.24 0.05 0.05
query4 1.68 0.08 0.08
query5 0.50 0.51 0.50
query6 1.13 0.72 0.73
query7 0.02 0.02 0.01
query8 0.05 0.05 0.04
query9 0.54 0.48 0.48
query10 0.53 0.53 0.54
query11 0.16 0.12 0.11
query12 0.16 0.13 0.12
query13 0.59 0.58 0.61
query14 0.76 0.77 0.77
query15 0.82 0.81 0.82
query16 0.35 0.35 0.37
query17 0.95 0.96 1.01
query18 0.23 0.23 0.25
query19 1.82 1.71 1.70
query20 0.01 0.02 0.01
query21 15.44 0.75 0.66
query22 4.29 7.22 1.88
query23 18.37 1.41 1.31
query24 1.45 0.37 0.21
query25 0.13 0.09 0.08
query26 0.26 0.17 0.18
query27 0.08 0.09 0.07
query28 13.36 1.01 1.02
query29 13.19 3.24 3.23
query30 0.24 0.06 0.06
query31 2.86 0.39 0.39
query32 3.32 0.48 0.47
query33 2.93 2.89 2.96
query34 16.93 4.46 4.46
query35 4.50 4.57 4.53
query36 0.65 0.46 0.45
query37 0.18 0.16 0.15
query38 0.15 0.15 0.14
query39 0.04 0.04 0.03
query40 0.16 0.15 0.15
query41 0.10 0.04 0.05
query42 0.06 0.05 0.05
query43 0.04 0.04 0.03
Total cold run time: 109.4 s
Total hot run time: 30.58 s