Lari Hotari

The impact of this bug is that the Netty defaults have been used (low 32kB, high 64kB) instead of the expected defaults low 384kB high 512kB.

I did some investigation on this issue and I found a pattern. The problem occurs if there are pending messages in the subscription. As described above, I first produced 6...

this is another example where there's multiple individual message ranges: ```json { "entriesAddedCounter" : 38551, "numberOfEntries" : 38551, "totalSize" : 3907922, "currentLedgerEntries" : 38551, "currentLedgerSize" : 3907922, "lastLedgerCreatedTimestamp" : "2021-03-31T07:34:58.097Z",...

I also wonder about the `messagesConsumedCounter`. I've only produced and consumed

another observation. After leaving it running, there are negative values in the messagesConsumedCounter: ```json { "entriesAddedCounter" : 0, "numberOfEntries" : 48221, "totalSize" : 4931008, "currentLedgerEntries" : 0, "currentLedgerSize" : 0,...

It seems that `org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl#getPreviousPosition` doesn't take the "marker messages" for replicated subscriptions into account and that is part of the problem. https://github.com/apache/pulsar/blob/1b8da9245e111718acb337bce310ffdef7b30496/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L3092-L3115

In PIP-33 there is [this documented limitation](https://github.com/apache/pulsar/wiki/PIP-33%3A-Replicated-subscriptions#limitations): > For this proposal, we're only targeting to sync the "mark-delete" position (eg: offset), without considering the messages deleted out of order after...

this test https://github.com/apache/pulsar/pull/10098/commits/28faa8efd98a88004222d810407875c585cd8d67 reproduces the issue when "allowDuplicates" is set to "false". The test is part of PR #10098 .

@merlimat do you have a chance to take a look at this issue and answer [the question above](https://github.com/apache/pulsar/issues/10054#issuecomment-811117125)? thanks the test case to reproduce the issue has been merged to...

> To "fix" the test I just put a `Thread.sleep(2 * config1.getReplicatedSubscriptionsSnapshotFrequencyMillis())` before the producer gets started. > > One other minor thing in the test is that we don't...