pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

Skip topics with remote replication producers in topic inactivity check

Open lhotari opened this issue 3 years ago • 7 comments

Motivation

  • currently there's a problem that the replicators get shutdown in the inactivity check
  • this is an improved fix for the issue described in #11382

Modifications

  • skip deleting a topic that has connected remote replication producers.
  • move existing check to happen before replicators are closed.

Additional context

An alternative solution is to disable delete-while-inactive for namespaces that are replicated.

pulsar-admin namespaces set-inactive-topic-policies tenant/namespace --disable-delete-while-inactive

lhotari avatar Jul 15 '22 09:07 lhotari

Question:

  1. Do you mean we will delete the topic at the second GC check?
  2. I'm not sure If it's a breaking change for the user. Maybe we need to notice this change?
  3. This will cause the broker to have a large number of unused producers, right?

mattisonchao avatar Jul 16 '22 01:07 mattisonchao

Question:

  1. Do you mean we will delete the topic at the second GC check?

@mattisonchao please see how the existing code works in https://github.com/apache/pulsar/blob/c48a3243287c7d775459b6437d9f4b24ed44cf4c/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L2182-L2231 . The existing logic closes replicators that contain no backlog. However if there are connected replication producers, the inactivity deletion will be skipped eventually and it's possible that some replicators were already closed and they won't be restored. This is why this PR is made since it's better to make the check as the first step instead of possibly closing some replicators and then terminating the inactivity deletion.

  1. I'm not sure If it's a breaking change for the user. Maybe we need to notice this change?

This PR doesn't introduce a breaking change.

  1. This will cause the broker to have a large number of unused producers, right?

No.

lhotari avatar Jul 16 '22 07:07 lhotari

@lhotari I got it, thanks for your explanation.

mattisonchao avatar Jul 16 '22 14:07 mattisonchao

@merlimat @codelipenghui Please review

lhotari avatar Aug 11 '22 07:08 lhotari

@merlimat @codelipenghui @Technoboy- Ping

Jason918 avatar Aug 27 '22 08:08 Jason918

The pr had no activity for 30 days, mark with Stale label.

github-actions[bot] avatar Sep 30 '22 02:09 github-actions[bot]

@lhotari Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

github-actions[bot] avatar Feb 01 '23 03:02 github-actions[bot]