starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Enhancement] Print compaction task statistics logs for cloud native table

Open wxl24life opened this issue 2 years ago • 3 comments

Why I'm doing: What I'm doing:

Record lake compaction process metrics, after this pr we can see compaction metrics in be/cn's log

I0226 15:05:07.353010 307673 vertical_compaction_task.cpp:109] Vertical compaction finished. tablet: 10089, txn_id: 10089, statistics: {"reader_total_time_ms":342033,"reader_io_ms":7424,"reader_io_count_remote":0,"reader_io_count_local_disk":1217774,"compressed_bytes_read":19553793796,"segment_init_ms":277,"column_iterator_init_ms":191,"segment_write_ms":586577}

Fixes #issue

What type of PR is this:

  • [ ] BugFix
  • [ ] Feature
  • [x] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [x] Yes, this PR will result in a change in behavior.
  • [ ] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [x] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [x] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 3.2
    • [x] 3.1
    • [ ] 3.0
    • [ ] 2.5

wxl24life avatar Dec 22 '23 03:12 wxl24life

[FE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Mar 01 '24 04:03 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 80 / 91 (87.91%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: be/src/storage/olap_common.h 0 1 00.00% [308]
:large_blue_circle: be/src/storage/lake/compaction_task_context.h 1 7 14.29% [34, 60, 62, 65, 66, 67]
:large_blue_circle: be/src/storage/lake/horizontal_compaction_task.cpp 14 16 87.50% [74, 79]
:large_blue_circle: be/src/storage/lake/vertical_compaction_task.cpp 14 16 87.50% [182, 187]
:large_blue_circle: be/src/storage/lake/vertical_compaction_task.h 2 2 100.00% []
:large_blue_circle: be/src/storage/lake/compaction_task.cpp 5 5 100.00% []
:large_blue_circle: be/src/storage/lake/compaction_scheduler.cpp 2 2 100.00% []
:large_blue_circle: be/src/storage/lake/tablet_writer.h 1 1 100.00% []
:large_blue_circle: be/src/storage/lake/tablet_manager.cpp 4 4 100.00% []
:large_blue_circle: be/src/storage/lake/compaction_task_context.cpp 30 30 100.00% []
:large_blue_circle: be/src/storage/lake/horizontal_compaction_task.h 2 2 100.00% []
:large_blue_circle: be/src/storage/lake/general_tablet_writer.cpp 5 5 100.00% []

github-actions[bot] avatar Mar 01 '24 04:03 github-actions[bot]

@Mergifyio backport branch-3.2

github-actions[bot] avatar Mar 01 '24 07:03 github-actions[bot]

@Mergifyio backport branch-3.1

github-actions[bot] avatar Mar 01 '24 07:03 github-actions[bot]

backport branch-3.2

✅ Backports have been created

mergify[bot] avatar Mar 01 '24 07:03 mergify[bot]

backport branch-3.1

✅ Backports have been created

mergify[bot] avatar Mar 01 '24 07:03 mergify[bot]