[fix][broker]Fix incorrect backlog if use multiple acknowledge types on the same subscription
Motivation
Background
By default, the backlog of a subscription is got by {num of entries published} - {num of entries acknowleged}
-
num of entries published:ManagedLedger.entriesAddedCounter -
num of entries acknowleged:ManagedLedger.messagesConsumedCounter
How it being modified when calling reset curosr or seek or cumulative acknowledge
- Calculate entries between
{original md-position}and{target md-position}, getscounter-1. If{target md-position}is smaller than{original md-position}, the result ofcounter-1will be negative. - Calculate entries that are before
{target md-position}and were acknowledged before callingreset curosr, getscounter-2. If{target md-position}is smaller than{original md-position}, the result ofcounter-2will be0. - Let
cursor.messagesConsumedCounteradd{counter-1} - {counter-2}
Issue 1: clears individual acknowledge status before calculate how many entries acked.
- Call
reset cursororseekorcumulative acknowledge - Broker moves
md-positionto the target. - Broker clears
cursor. individualDeletedMessages, see https://github.com/apache/pulsar/blob/v4.1.2/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java#L2315-L2316 - Broker calculate how many entries that were acked, see Background for the implementation.
- Issue: the items that mantained by
cursor. individualDeletedMessageshas been removed at setp 3. a. The step 4 can not get a correct{counter-2}, which is described at the Background.
Issue 2: reset curosr or seek forgtes to reduce the counter that acknowledged messages
- Status:
md-positiin : 3:5,cursor. individualDeletedMessages:[3:10 ~ 3:11] - Call
reset cursororseekto3: 5, which is the same as before. - Broker clears
cursor. individualDeletedMessages, but forget to reducecursor.messagesConsumedCounterthatcursor. individualDeletedMessages:[3:10 ~ 3:11]increased.
Modifications
- fix bugs
Documentation
- [ ]
doc - [ ]
doc-required - [x]
doc-not-needed - [ ]
doc-complete
Matching PR in forked repository
PR in forked repository: x
/pulsarbot rerun-failure-checks
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 74.40%. Comparing base (9d8bf60) to head (f295547).
:warning: Report is 10 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #25047 +/- ##
============================================
- Coverage 74.48% 74.40% -0.09%
- Complexity 34174 34176 +2
============================================
Files 1921 1921
Lines 150380 150436 +56
Branches 17471 17471
============================================
- Hits 112009 111930 -79
- Misses 29483 29613 +130
- Partials 8888 8893 +5
| Flag | Coverage Δ | |
|---|---|---|
| inttests | 26.37% <76.36%> (-0.35%) |
:arrow_down: |
| systests | 22.86% <65.45%> (-0.10%) |
:arrow_down: |
| unittests | 73.93% <100.00%> (-0.08%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Files with missing lines | Coverage Δ | |
|---|---|---|
| ...che/bookkeeper/mledger/impl/ManagedCursorImpl.java | 78.62% <100.00%> (-0.47%) |
:arrow_down: |
| .../bookkeeper/mledger/impl/NonDurableCursorImpl.java | 81.48% <100.00%> (+0.34%) |
:arrow_up: |
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.