[Enhancement] Print compaction task statistics logs for cloud native table
Why I'm doing: What I'm doing:
Record lake compaction process metrics, after this pr we can see compaction metrics in be/cn's log
I0226 15:05:07.353010 307673 vertical_compaction_task.cpp:109] Vertical compaction finished. tablet: 10089, txn_id: 10089, statistics: {"reader_total_time_ms":342033,"reader_io_ms":7424,"reader_io_count_remote":0,"reader_io_count_local_disk":1217774,"compressed_bytes_read":19553793796,"segment_init_ms":277,"column_iterator_init_ms":191,"segment_write_ms":586577}
Fixes #issue
What type of PR is this:
- [ ] BugFix
- [ ] Feature
- [x] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [x] Yes, this PR will result in a change in behavior.
- [ ] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [x] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [x] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [ ] This is a backport pr
Bugfix cherry-pick branch check:
- [x] I have checked the version labels which the pr will be auto-backported to the target branch
- [x] 3.2
- [x] 3.1
- [ ] 3.0
- [ ] 2.5
Quality Gate passed
Issues
0 New issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
[FE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
[BE Incremental Coverage Report]
:white_check_mark: pass : 80 / 91 (87.91%)
file detail
| path | covered_line | new_line | coverage | not_covered_line_detail | |
|---|---|---|---|---|---|
| :large_blue_circle: | be/src/storage/olap_common.h | 0 | 1 | 00.00% | [308] |
| :large_blue_circle: | be/src/storage/lake/compaction_task_context.h | 1 | 7 | 14.29% | [34, 60, 62, 65, 66, 67] |
| :large_blue_circle: | be/src/storage/lake/horizontal_compaction_task.cpp | 14 | 16 | 87.50% | [74, 79] |
| :large_blue_circle: | be/src/storage/lake/vertical_compaction_task.cpp | 14 | 16 | 87.50% | [182, 187] |
| :large_blue_circle: | be/src/storage/lake/vertical_compaction_task.h | 2 | 2 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/lake/compaction_task.cpp | 5 | 5 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/lake/compaction_scheduler.cpp | 2 | 2 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/lake/tablet_writer.h | 1 | 1 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/lake/tablet_manager.cpp | 4 | 4 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/lake/compaction_task_context.cpp | 30 | 30 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/lake/horizontal_compaction_task.h | 2 | 2 | 100.00% | [] |
| :large_blue_circle: | be/src/storage/lake/general_tablet_writer.cpp | 5 | 5 | 100.00% | [] |
@Mergifyio backport branch-3.2
@Mergifyio backport branch-3.1
backport branch-3.2
✅ Backports have been created
- #41972 [Enhancement] Print compaction task statistics logs for cloud native table (backport #37616) has been created for branch
branch-3.2
backport branch-3.1
✅ Backports have been created
- #41973 [Enhancement] Print compaction task statistics logs for cloud native table (backport #37616) has been created for branch
branch-3.1but encountered conflicts