pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix] [broker] Disabled deduplication, but ledger-store will not be d…

Open poorbarcode opened this issue 3 years ago • 13 comments

Motivation

When execute command ulsar-admin topicPolicies get-deduplication persistent://{tenant}/{ns}/{topic} failure.

Then the deduplication task will be disabled, and cursor 'pulsar.dedup' will not move still restart broker.

But zk-node: /managed-ledgers/{tenant}/{ns}/persistent/{topic}/pulsar.dedup still exists.

Errors occur: this topic's ledger-store will not be deleted anymore, even if all message has been acknowledged.

Modifications

  • When checkDeduplicationStatus() failure, stop this topic.
  • Rewrite the state change of MessageDeduplication

Documentation

Need to update docs?

  • [ ] doc-required

  • [x] no-need-doc

  • [ ] doc

  • [ ] doc-added

poorbarcode avatar Apr 28 '22 07:04 poorbarcode

Another question: Can we add tests to verify this issue?

mattisonchao avatar Apr 28 '22 07:04 mattisonchao

Little question: I'm not sure if the issue is unrecoverable, but it feels a little bad to close the topic. Please let me know what you think.

Another question: Can we add tests to verify this issue?

OK. Thank you for reminding me. liang zai

poorbarcode avatar Apr 28 '22 09:04 poorbarcode

Little question: I'm not sure if the issue is unrecoverable, but it feels a little bad to close the topic. Please let me know what you think.

I'm not sure if the issue is unrecoverable, but it feels a little bad to close the topic.

If deduplication can't work properly, means topic can't work properly.

When user changes deduplication status, it should be clear that works or not.

Do close topic will rerty start deduplication task or told user topic was done, both of these results are useful to user

poorbarcode avatar Apr 28 '22 09:04 poorbarcode

/pulsarbot rerun-failure-checks

poorbarcode avatar Apr 30 '22 06:04 poorbarcode

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar Jun 04 '22 02:06 github-actions[bot]

/pulsarbot rerun-failure-checks

poorbarcode avatar Jun 13 '22 12:06 poorbarcode

/pulsarbot rerun-failure-checks

poorbarcode avatar Jun 13 '22 18:06 poorbarcode

@poorbarcode Please provide a correct documentation label for your PR. Instructions see Pulsar Documentation Label Guide.

github-actions[bot] avatar Jun 30 '22 05:06 github-actions[bot]

@poorbarcode Please provide a correct documentation label for your PR. Instructions see Pulsar Documentation Label Guide.

github-actions[bot] avatar Jun 30 '22 05:06 github-actions[bot]

@poorbarcode Please provide a correct documentation label for your PR. Instructions see Pulsar Documentation Label Guide.

github-actions[bot] avatar Jun 30 '22 05:06 github-actions[bot]

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar Sep 27 '22 02:09 github-actions[bot]

@poorbarcode hi, I move this PR to release/2.9.5, if you have any questions, please ping me. thanks.

congbobo184 avatar Nov 17 '22 12:11 congbobo184

As discussed on the mailing list https://lists.apache.org/thread/w4jzk27qhtosgsz7l9bmhf1t7o9mxjhp, there is no plan to release 2.9.6, so I am going to remove the release/2.9.6 label

michaeljmarshall avatar Jun 27 '23 21:06 michaeljmarshall