pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix] prevent offload non-persistent or system topics

Open chunxiaozheng opened this issue 9 months ago • 5 comments

When ledger is closed, do not offload when this ledger is non-persistent or system topics.

Motivation

When ledger is closed, do not offload when this ledger is non-persistent or system topics.

Modifications

Verifying this change

  • [ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • [ ] Dependencies (add or upgrade a dependency)
  • [ ] The public API
  • [ ] The schema
  • [ ] The default values of configurations
  • [ ] The threading model
  • [ ] The binary protocol
  • [ ] The REST endpoints
  • [ ] The admin CLI options
  • [ ] The metrics
  • [ ] Anything that affects deployment

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository:

chunxiaozheng avatar Apr 29 '24 02:04 chunxiaozheng

NonPersistentTopic has no ledgers to persist messages, so we don't need to care about it. About SystemTopic, I remember there is a PR to fix it

dao-jun avatar Apr 29 '24 02:04 dao-jun

Got it, https://github.com/apache/pulsar/pull/22497

dao-jun avatar Apr 29 '24 02:04 dao-jun

Got it, #22497

Thanks! But I see the #22497 did not merge. In our scenario, due to system topic triggered offload, many exceptions were printed in the log, so I submitted the above modifications

chunxiaozheng avatar Apr 29 '24 03:04 chunxiaozheng

Got it, #22497

Thanks! But I see the #22497 did not merge. In our scenario, due to system topic triggered offload, many exceptions were printed in the log, so I submitted the above modifications

let me ask him, if there is no other problems, I'll reopen and merge it

dao-jun avatar Apr 29 '24 03:04 dao-jun

Got it, #22497

Thanks! But I see the #22497 did not merge. In our scenario, due to system topic triggered offload, many exceptions were printed in the log, so I submitted the above modifications

let me ask him, if there is no other problems, I'll reopen and merge it

Thanks!

chunxiaozheng avatar Apr 29 '24 03:04 chunxiaozheng

@chunxiaozheng https://github.com/apache/pulsar/pull/22497 merged, let's close this PR

dao-jun avatar May 08 '24 05:05 dao-jun

@chunxiaozheng #22497 merged, let's close this PR

okay,thanks!

chunxiaozheng avatar May 09 '24 07:05 chunxiaozheng