pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix][broker] Set ServiceUnitStateChannel topic compaction threshold explicitly, improve getOwnerAsync, and fix other bugs

Open heesung-sohn opened this issue 1 year ago • 0 comments

Motivation

We better set the compaction threshold of the serviceUnitStateChannel topic instead of relying on other components' init.

Also, we need to fix the issue that lookup and unload often timeout with 500 error.

Modifications

  • Set the compaction threshold of the serviceUnitStateChannel topic in the monitor thread
  • Release metadataAndPayload buffer in RawBatchMessageContainerImpl.
  • Automatically restart load data store producer and table view if inactive.
  • Updated getOwnerAsync for better synchronization of ownership checks, deferred lookups and active broker checks
  • added minor retry logic in the ExtensibleLoadManager tests

Verifying this change

  • [x] Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • [ ] Dependencies (add or upgrade a dependency)
  • [ ] The public API
  • [ ] The schema
  • [ ] The default values of configurations
  • [ ] The threading model
  • [ ] The binary protocol
  • [ ] The REST endpoints
  • [ ] The admin CLI options
  • [ ] The metrics
  • [ ] Anything that affects deployment

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository: https://github.com/heesung-sn/pulsar/pull/61

heesung-sohn avatar Feb 17 '24 05:02 heesung-sohn