doris
doris copied to clipboard
[Bug](profile) fix the BytesScannedFromCache profile always is 0
What problem does this PR solve?
Problem Summary: the cache_profile will call update in _collect_profile_before_close() function, so couldn't reset about bytes_read_from_local/remote fields.
void FileScanner::_collect_profile_before_close() {
Scanner::_collect_profile_before_close();
if (config::enable_file_cache && _state->query_options().enable_file_cache &&
_profile != nullptr) {
io::FileCacheProfileReporter cache_profile(_profile);
cache_profile.update(_file_cache_statistics.get());
_state->get_query_ctx()->resource_ctx()->io_context()->update_bytes_write_into_cache(
_file_cache_statistics->bytes_write_into_cache);
}
Release note
None
Check List (For Author)
-
Test
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
-
Behavior changed:
- [ ] No.
- [ ] Yes.
-
Does this need documentation?
- [ ] No.
- [ ] Yes.
Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
Thank you for your contribution to Apache Doris. Don't know what should be done next? See How to process your PR.
Please clearly describe your PR:
- What problem was fixed (it's best to include specific error reporting information). How it was fixed.
- Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
- What features were added. Why was this function added?
- Which code was refactored and why was this part of the code refactored?
- Which functions were optimized and what is the difference before and after the optimization?
run buildall
TPC-H: Total hot run time: 36607 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 59bbc637e365e843fd32f24f872a3a44714ace80, data reload: false
------ Round 1 ----------------------------------
q1 17605 4219 4077 4077
q2 2009 348 226 226
q3 10186 1366 733 733
q4 10232 897 314 314
q5 7528 2122 1931 1931
q6 184 165 136 136
q7 1020 854 717 717
q8 9351 1402 1197 1197
q9 7028 5283 5374 5283
q10 6846 2380 1953 1953
q11 504 317 293 293
q12 665 719 588 588
q13 18220 3655 3103 3103
q14 303 325 283 283
q15 596 513 515 513
q16 940 942 888 888
q17 716 855 511 511
q18 7553 7977 7742 7742
q19 1878 1058 638 638
q20 407 394 259 259
q21 4555 4213 4212 4212
q22 1141 1083 1010 1010
Total cold run time: 109467 ms
Total hot run time: 36607 ms
----- Round 2, with runtime_filter_mode=off -----
q1 4362 4293 4241 4241
q2 336 499 345 345
q3 2593 2777 2405 2405
q4 1410 1846 1401 1401
q5 4631 4466 4462 4462
q6 216 164 123 123
q7 2169 1897 1766 1766
q8 2623 2495 2640 2495
q9 7569 7418 7446 7418
q10 2879 3099 2645 2645
q11 564 497 483 483
q12 645 671 537 537
q13 3220 3655 2989 2989
q14 257 290 256 256
q15 531 494 496 494
q16 850 885 834 834
q17 1115 1348 1368 1348
q18 7281 7009 6962 6962
q19 825 811 812 811
q20 1897 1975 1819 1819
q21 4656 4268 4172 4172
q22 1121 999 985 985
Total cold run time: 51750 ms
Total hot run time: 48991 ms
TPC-DS: Total hot run time: 181008 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 59bbc637e365e843fd32f24f872a3a44714ace80, data reload: false
query5 5062 628 462 462
query6 326 240 209 209
query7 4222 467 277 277
query8 322 250 230 230
query9 8775 2562 2565 2562
query10 541 382 318 318
query11 15226 14772 14882 14772
query12 181 114 111 111
query13 1250 508 401 401
query14 6276 3266 3009 3009
query14_1 2936 2899 2878 2878
query15 208 202 186 186
query16 973 484 449 449
query17 1129 723 603 603
query18 2548 449 357 357
query19 235 263 209 209
query20 122 117 111 111
query21 219 140 121 121
query22 3899 3969 4015 3969
query23 16491 16162 15986 15986
query23_1 15982 15962 16059 15962
query24 7395 1688 1276 1276
query24_1 1248 1220 1249 1220
query25 592 500 450 450
query26 1261 276 166 166
query27 2750 471 323 323
query28 4469 2161 2144 2144
query29 814 582 466 466
query30 315 251 212 212
query31 854 685 631 631
query32 90 69 68 68
query33 557 344 302 302
query34 904 918 549 549
query35 809 804 730 730
query36 890 956 814 814
query37 128 96 135 96
query38 3848 3852 3764 3764
query39 757 730 719 719
query39_1 698 707 698 698
query40 223 140 120 120
query41 66 67 62 62
query42 107 103 109 103
query43 434 430 410 410
query44 1371 760 759 759
query45 195 189 185 185
query46 891 981 630 630
query47 1657 1687 1626 1626
query48 319 330 244 244
query49 667 425 354 354
query50 670 296 220 220
query51 3840 3807 3797 3797
query52 113 119 98 98
query53 328 371 298 298
query54 295 261 255 255
query55 80 78 72 72
query56 306 288 299 288
query57 1150 1142 1096 1096
query58 263 251 250 250
query59 2465 2443 2325 2325
query60 308 315 293 293
query61 201 153 157 153
query62 721 702 625 625
query63 324 298 299 298
query64 4930 1299 988 988
query65 4023 3971 3912 3912
query66 1419 444 322 322
query67 15087 14912 14695 14695
query68 8334 1042 738 738
query69 494 346 308 308
query70 1084 977 980 977
query71 366 315 280 280
query72 6038 4950 5051 4950
query73 660 568 316 316
query74 8833 8933 8588 8588
query75 3606 3538 3152 3152
query76 3944 1136 757 757
query77 528 394 297 297
query78 9406 9568 8847 8847
query79 1443 888 601 601
query80 733 655 554 554
query81 512 268 229 229
query82 203 131 111 111
query83 269 256 241 241
query84 265 116 95 95
query85 913 497 449 449
query86 386 288 293 288
query87 4020 4091 3982 3982
query88 3178 2277 2278 2277
query89 456 429 396 396
query90 2164 165 159 159
query91 169 168 140 140
query92 86 68 63 63
query93 2036 917 571 571
query94 475 313 285 285
query95 578 373 299 299
query96 594 464 211 211
query97 2568 2648 2542 2542
query98 213 197 194 194
query99 1305 1269 1216 1216
Total cold run time: 264073 ms
Total hot run time: 181008 ms
ClickBench: Total hot run time: 27.27 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 59bbc637e365e843fd32f24f872a3a44714ace80, data reload: false
query1 0.06 0.05 0.05
query2 0.10 0.05 0.05
query3 0.26 0.08 0.08
query4 1.61 0.12 0.10
query5 0.28 0.27 0.28
query6 1.17 0.64 0.62
query7 0.04 0.03 0.03
query8 0.05 0.04 0.04
query9 0.57 0.51 0.50
query10 0.57 0.56 0.55
query11 0.16 0.11 0.12
query12 0.15 0.12 0.11
query13 0.63 0.61 0.60
query14 0.98 0.98 0.99
query15 0.82 0.82 0.79
query16 0.42 0.41 0.39
query17 1.10 1.09 1.04
query18 0.24 0.21 0.21
query19 1.97 1.86 1.83
query20 0.02 0.02 0.01
query21 15.42 0.30 0.14
query22 4.83 0.06 0.04
query23 16.08 0.28 0.10
query24 1.60 1.26 0.29
query25 0.10 0.08 0.05
query26 0.15 0.14 0.14
query27 0.07 0.06 0.05
query28 4.26 1.23 1.02
query29 12.59 3.97 3.22
query30 0.27 0.13 0.12
query31 2.83 0.63 0.40
query32 3.23 0.56 0.45
query33 3.05 2.99 3.02
query34 16.90 5.17 4.53
query35 4.58 4.63 4.54
query36 0.68 0.50 0.49
query37 0.10 0.06 0.06
query38 0.07 0.04 0.03
query39 0.05 0.03 0.03
query40 0.17 0.15 0.13
query41 0.09 0.03 0.04
query42 0.04 0.02 0.03
query43 0.04 0.04 0.04
Total cold run time: 98.4 s
Total hot run time: 27.27 s
BE UT Coverage Report
Increment line coverage :tada:
Increment coverage report Complete coverage report
| Category | Coverage |
|---|---|
| Function Coverage | 53.42% (18822/35237) |
| Line Coverage | 39.16% (174036/444401) |
| Region Coverage | 33.83% (134855/398641) |
| Branch Coverage | 34.75% (58008/166910) |