pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

Published 20 hours ago • verified publisher icon apache

[Fix][broker] Limit replication rate based on bytes

Open nodece opened this issue 1 year ago • 2 comments

Motivation

When the replicator rate limit value based on bytes is less than the org.apache.pulsar.broker.service.persistent.PersistentReplicator#readMaxSizeBytes, this will cause the outgoing bytes to exceed the limit value.

The goal is to make outgoing bytes as close to the rate limit value as possible.

Modifications

  • org.apache.pulsar.broker.service.persistent.PersistentReplicator#getAvailablePermits returns the available messages and bytes, and then reads the entries based on that.

Verifying this change

The test has been added.

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

nodece avatar May 08 '24 10:05 nodece

/pulsarbot rerun-failure-checks

nodece avatar May 13 '24 18:05 nodece

Codecov Report

Attention: Patch coverage is 65.38462% with 9 lines in your changes missing coverage. Please review.

Project coverage is 73.43%. Comparing base (bbc6224) to head (51c5c0d). Report is 427 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #22674      +/-   ##
============================================
- Coverage     73.57%   73.43%   -0.14%     
- Complexity    32624    33275     +651     
============================================
  Files          1877     1907      +30     
  Lines        139502   142961    +3459     
  Branches      15299    15582     +283     
============================================
+ Hits         102638   104985    +2347     
- Misses        28908    29945    +1037     
- Partials       7956     8031      +75
Flag Coverage Δ
inttests 27.74% <42.30%> (+3.16%) :arrow_up:
systests 24.75% <0.00%> (+0.43%) :arrow_up:
unittests 72.45% <65.38%> (-0.39%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...roker/service/persistent/PersistentReplicator.java 68.89% <65.38%> (+0.02%) :arrow_up:

... and 482 files with indirect coverage changes

codecov-commenter avatar May 13 '24 20:05 codecov-commenter

/pulsarbot rerun-failure-checks

nodece avatar Jun 14 '24 02:06 nodece