pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix][broker]Fix incorrect backlog if use multiple acknowledge types on the same subscription

Open poorbarcode opened this issue 1 month ago • 2 comments

Motivation

Background By default, the backlog of a subscription is got by {num of entries published} - {num of entries acknowleged}

  • num of entries published: ManagedLedger.entriesAddedCounter
  • num of entries acknowleged: ManagedLedger.messagesConsumedCounter

How it being modified when calling reset curosr or seek or cumulative acknowledge

  • Calculate entries between {original md-position} and {target md-position}, gets counter-1. If {target md-position} is smaller than {original md-position}, the result of counter-1 will be negative.
  • Calculate entries that are before {target md-position} and were acknowledged before calling reset curosr, gets counter-2. If {target md-position} is smaller than {original md-position}, the result of counter-2 will be 0.
  • Let cursor.messagesConsumedCounter add {counter-1} - {counter-2}

Issue 1: clears individual acknowledge status before calculate how many entries acked.

  1. Call reset cursor or seek or cumulative acknowledge
  2. Broker moves md-position to the target.
  3. Broker clears cursor. individualDeletedMessages, see https://github.com/apache/pulsar/blob/v4.1.2/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java#L2315-L2316
  4. Broker calculate how many entries that were acked, see Background for the implementation.
  5. Issue: the items that mantained by cursor. individualDeletedMessages has been removed at setp 3. a. The step 4 can not get a correct {counter-2}, which is described at the Background.

Issue 2: reset curosr or seek forgtes to reduce the counter that acknowledged messages

  1. Status: md-positiin : 3:5, cursor. individualDeletedMessages: [3:10 ~ 3:11]
  2. Call reset cursor or seek to 3: 5, which is the same as before.
  3. Broker clears cursor. individualDeletedMessages, but forget to reduce cursor.messagesConsumedCounter that cursor. individualDeletedMessages: [3:10 ~ 3:11] increased.

Modifications

  • fix bugs

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository: x

poorbarcode avatar Dec 08 '25 05:12 poorbarcode

/pulsarbot rerun-failure-checks

poorbarcode avatar Dec 08 '25 05:12 poorbarcode

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 74.40%. Comparing base (9d8bf60) to head (f295547). :warning: Report is 10 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #25047      +/-   ##
============================================
- Coverage     74.48%   74.40%   -0.09%     
- Complexity    34174    34176       +2     
============================================
  Files          1921     1921              
  Lines        150380   150436      +56     
  Branches      17471    17471              
============================================
- Hits         112009   111930      -79     
- Misses        29483    29613     +130     
- Partials       8888     8893       +5     
Flag Coverage Δ
inttests 26.37% <76.36%> (-0.35%) :arrow_down:
systests 22.86% <65.45%> (-0.10%) :arrow_down:
unittests 73.93% <100.00%> (-0.08%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...che/bookkeeper/mledger/impl/ManagedCursorImpl.java 78.62% <100.00%> (-0.47%) :arrow_down:
.../bookkeeper/mledger/impl/NonDurableCursorImpl.java 81.48% <100.00%> (+0.34%) :arrow_up:

... and 102 files with indirect coverage changes

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

codecov-commenter avatar Dec 10 '25 07:12 codecov-commenter