starrocks icon indicating copy to clipboard operation
starrocks copied to clipboard

[Refactor] Refactor cross-cluster replication limits

Open xiangguangyxg opened this issue 1 year ago • 3 comments

Why I'm doing:

Some limits of cross-cluster replication are inappropriate.

What I'm doing:

Add config replication_max_parallel_replica_count with default value 10240. Change the default value of replication_max_parallel_data_size_mb from 10G to 1T. Change the default value of replication_threads from number of cpu cores to 4 times of number of cpu cores.

Fixes #issue

What type of PR is this:

  • [ ] BugFix
  • [ ] Feature
  • [ ] Enhancement
  • [x] Refactor
  • [ ] UT
  • [ ] Doc
  • [ ] Tool

Does this PR entail a change in behavior?

  • [ ] Yes, this PR will result in a change in behavior.
  • [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • [ ] Parameter changes: default values, similar parameters but with different default values
  • [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
  • [ ] Feature removed
  • [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • [ ] I have added test cases for my bug fix or my new feature
  • [x] This pr needs user documentation (for new or modified features or behaviors)
    • [x] I have added documentation for my new feature or new function
  • [ ] This is a backport pr

Bugfix cherry-pick branch check:

  • [x] I have checked the version labels which the pr will be auto-backported to the target branch
    • [x] 3.3
    • [x] 3.2
    • [x] 3.1
    • [ ] 3.0
    • [ ] 2.5

xiangguangyxg avatar May 11 '24 07:05 xiangguangyxg

Quality Gate Failed Quality Gate failed

Failed conditions
9.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud

sonarqubecloud[bot] avatar May 11 '24 07:05 sonarqubecloud[bot]

[FE Incremental Coverage Report]

:white_check_mark: pass : 41 / 46 (89.13%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/replication/ReplicationMgr.java 5 8 62.50% [95, 193, 194]
:large_blue_circle: com/starrocks/replication/ReplicationJob.java 33 35 94.29% [114, 399]
:large_blue_circle: com/starrocks/common/Config.java 3 3 100.00% []

github-actions[bot] avatar May 11 '24 08:05 github-actions[bot]

[BE Incremental Coverage Report]

:x: fail : 4 / 7 (57.14%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: be/src/agent/agent_server.cpp 4 7 57.14% [74, 77, 590]

github-actions[bot] avatar May 11 '24 08:05 github-actions[bot]

@Mergify backport branch-3.3 branch-3.2 branch-3.1

xiangguangyxg avatar May 14 '24 02:05 xiangguangyxg

backport branch-3.3 branch-3.2 branch-3.1

✅ Backports have been created

mergify[bot] avatar May 14 '24 02:05 mergify[bot]