pulsar
pulsar copied to clipboard
[improve] [broker] improve the restart broker can load bundle as one of the best brokers when selectBroker in LeastResourceUsageWithWeight
Motivation
As the brokerAvgResourceUsageWithWeight
stored the broker's history resource usage, if a broker is high load before restart, after the broker restart always show a high load history resource usage, and loadBalancerHistoryResourcePercentage = 0.9
, which will cause the first few times the broker can not be select as best broker to load bundles. Only when selectBrokerForAssignment
invoke few time's , the brokerAvgResourceUsageWithWeight
could be neutralized by the current resourceUsage.
But bundles unload is not a frequently operator if not disable doLoadShedding
,it will wait a long time; So if a broker restarted we can clean the brokers history resource usage to guaranty the no bundles or no msgRate broker can be select as the one of best brokers.
https://github.com/apache/pulsar/blob/bc94643bc1a7f365dfb75e389ac3e1156770a119/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/LeastResourceUsageWithWeight.java#L99-L100
Modifications
- If the broker not own bundles or the msgRate is 0, use current resourceUsage to cover the historyUsage.
- add some unit test.
Documentation
- [X]
doc-not-needed
(Please explain why)
/pulsarbot run-failure-checks
The pr had no activity for 30 days, mark with Stale label.
/pulsarbot run-failure-checks
Codecov Report
Merging #16766 (3c0c603) into master (99e26f5) will increase coverage by
0.41%
. The diff coverage is0.00%
.
@@ Coverage Diff @@
## master #16766 +/- ##
============================================
+ Coverage 44.79% 45.20% +0.41%
- Complexity 10026 10817 +791
============================================
Files 703 763 +60
Lines 68808 73575 +4767
Branches 7375 7912 +537
============================================
+ Hits 30822 33260 +2438
- Misses 34378 36592 +2214
- Partials 3608 3723 +115
Flag | Coverage Δ | |
---|---|---|
unittests | 45.20% <0.00%> (+0.41%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
Impacted Files | Coverage Δ | |
---|---|---|
...loadbalance/impl/LeastResourceUsageWithWeight.java | 0.00% <0.00%> (ø) |
|
...pache/bookkeeper/mledger/impl/RangeSetWrapper.java | 43.39% <0.00%> (-2.61%) |
:arrow_down: |
...org/apache/bookkeeper/mledger/util/RangeCache.java | 67.04% <0.00%> (-2.28%) |
:arrow_down: |
...he/pulsar/broker/admin/v2/NonPersistentTopics.java | 60.64% <0.00%> (-1.39%) |
:arrow_down: |
...rg/apache/pulsar/client/impl/PulsarClientImpl.java | 43.85% <0.00%> (-0.95%) |
:arrow_down: |
...apache/pulsar/client/impl/AutoClusterFailover.java | 69.44% <0.00%> (-0.56%) |
:arrow_down: |
...he/pulsar/client/impl/MultiTopicsConsumerImpl.java | 22.86% <0.00%> (-0.12%) |
:arrow_down: |
...pulsar/metadata/impl/stats/MetadataStoreStats.java | 88.88% <0.00%> (ø) |
|
...g/apache/pulsar/metadata/api/NotificationType.java | 100.00% <0.00%> (ø) |
|
...ache/pulsar/metadata/api/MetadataStoreFactory.java | 0.00% <0.00%> (ø) |
|
... and 154 more |