pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[Bug] Topic compaction not working as expected and leading to ack holes on the subscription while read compacted true

Open sindhushreem25 opened this issue 1 year ago • 1 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Version

2.10.4

Minimal reproduce step

Create a sample producer which produces key based messages .like k1 value1 k2 value2 k1 "" k2 ""

create the sample consumer where trigger compaction and read with read compacted true .

value read will be k1 "" k2 ""

but when we see the internal stats ,observe the ack holes .

What did you expect to see?

ack holes should not be their on the subscriptions .

What did you see instead?

ack holes is seen on the subscription .

Anything else?

compaction.txt attached the sample produce and consume along with internal stats and stats for the topic .

Are you willing to submit a PR?

  • [ ] I'm willing to submit a PR!

sindhushreem25 avatar Jan 29 '24 12:01 sindhushreem25

For readCompacted=true, I think you should use acknowledgeCumulative to avoid ack holes.

coderzc avatar Jan 30 '24 14:01 coderzc

I think the same issue will appear on shared subscription, where cumulative ack cannot be used, no?

michalcukierman avatar Apr 01 '24 00:04 michalcukierman