pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix][broker] Disable system topic message deduplication

Open mattisonchao opened this issue 1 year ago • 0 comments

Motivation

Disable deduplication on system topic to avoid recovering deduplication WAL (especially from offloaded topic). Because the system topic usually is a precondition to other topics. Therefore, we should pay attention to topic loading time.

Note: If the system topic loading timeout may cause dependent topics to fail to run. Dependency diagram: normal topic --rely on--> system topic --rely on--> deduplication recover --may rely on--> (tiered storage)

Modifications

  • MessageDeduplication use Topic#isDeduplicationEnabled to judge if we should enable deduplication.
  • System topic overrides the Topic#isDeduplicationEnabled to return false.

Verifying this change

  • [x] Make sure that the change passes the CI checks.

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

mattisonchao avatar Apr 25 '24 07:04 mattisonchao