starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Enhancement] Default enable big broker load profile

Open meegoo opened this issue 1 year ago • 3 comments

Why I'm doing:

Broker load usually has long runtimes and often encounters performance issues that need to be identified. Therefore, automatically enabling profiling in conjunction with runtime profiles can greatly enhance the efficiency of problem identification. For insert into, we have not enabled profiling for big queries by default because we have encountered many complex insert into execution plans in customer scenarios that consume a significant amount of FE memory. Broker load does not have this issue as its execution plans are simple and include concurrency control.

What I'm doing:

Fixes #issue

What type of PR is this:

  • [ ] BugFix
  • [ ] Feature
  • [x] Enhancement
  • [ ] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [x] I have added test cases for my bug fix or my new feature
  • [ ] This pr needs user documentation (for new or modified features or behaviors)
    • [ ] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 3.3
    • [x] 3.2
    • [x] 3.1
    • [ ] 3.0
    • [ ] 2.5

meegoo avatar Apr 24 '24 13:04 meegoo

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

sonarqubecloud[bot] avatar Apr 25 '24 13:04 sonarqubecloud[bot]

[FE Incremental Coverage Report]

:white_check_mark: pass : 36 / 42 (85.71%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/load/loadv2/LoadLoadingTask.java 0 3 00.00% [236, 237, 243]
:large_blue_circle: com/starrocks/load/streamload/StreamLoadTask.java 0 1 00.00% [1076]
:large_blue_circle: com/starrocks/common/util/RuntimeProfileParser.java 2 3 66.67% [116]
:large_blue_circle: com/starrocks/qe/DefaultCoordinator.java 3 4 75.00% [534]
:large_blue_circle: com/starrocks/qe/scheduler/QueryRuntimeProfile.java 3 3 100.00% []
:large_blue_circle: com/starrocks/common/Config.java 1 1 100.00% []
:large_blue_circle: com/starrocks/common/util/ProfileManager.java 26 26 100.00% []
:large_blue_circle: com/starrocks/qe/scheduler/dag/JobSpec.java 1 1 100.00% []

github-actions[bot] avatar Apr 26 '24 04:04 github-actions[bot]

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] avatar Apr 26 '24 04:04 github-actions[bot]

@Mergifyio backport branch-3.3

github-actions[bot] avatar May 16 '24 07:05 github-actions[bot]

@Mergifyio backport branch-3.2

github-actions[bot] avatar May 16 '24 07:05 github-actions[bot]

@Mergifyio backport branch-3.1

github-actions[bot] avatar May 16 '24 07:05 github-actions[bot]

backport branch-3.3

✅ Backports have been created

mergify[bot] avatar May 16 '24 07:05 mergify[bot]

backport branch-3.2

✅ Backports have been created

mergify[bot] avatar May 16 '24 07:05 mergify[bot]

backport branch-3.1

✅ Backports have been created

mergify[bot] avatar May 16 '24 07:05 mergify[bot]