pulsar icon indicating copy to clipboard operation
pulsar copied to clipboard

[fix] [broker] Fix can not subscribe partitioned topic with a suffix-matched regexp

Open poorbarcode opened this issue 1 year ago • 0 comments

Motivation

If a consumer tries to subscribe to a partitioned topic with a suffix-matched regexp, it does not work.

  • create topic persistent://public/default/tp
  • create a consumer with regexp persistent://public/default/tp$
  • the pattern consumer has no internal consumers.

You can reproduce the issue by the test testPreciseRegexpSubscribe.

  • it can work when using 2.10.x
  • it can not work when using >=2.11.0, it is a break change due to PIP-145

https://github.com/apache/pulsar/pull/21885 fixed one wrong logic here, but there is still another wrong logic, see https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/TopicListService.java#L64

@Override
public void accept(String topicName, NotificationType notificationType) {
    // The param "topicName" contains the partition suffix, which causes a suffix-matched regexp not to match.
    if (topicsPattern.matcher(topicName).matches()) {
        List<String> newTopics;
        List<String> deletedTopics;
    ...

Why can the test testPreciseRegexpSubscribe pass?

https://github.com/apache/pulsar/pull/21885 added a test testPreciseRegexpSubscribe that covers the test which creates a consumer with a suffix-matched regexp. It works as below:

  • create a consumer with a suffix-matched regexp
  • create a partitioned topic
  • two events will trigger the pattern consumer to auto-create an internal single partition consumer.
    • Start a new recheckTopicsChangeAfterReconnect after the topic watcher is connected. https://github.com/apache/pulsar/pull/21885 fixed the wrong behavior.
    • Receive an event that a new topic created. Current PR is trying to fix the wrong behavior.

The test testPreciseRegexpSubscribe can not pass as the below flow:

  • create a consumer with a suffix-matched regexp
  • wait for Topic List Watcher to start.
    • the consumer will call recheckTopicsChangeAfterReconnect at this step.
  • create a partitioned topic
    • the consumer can not receive the event due to the wrong logic, then the test failed.

Modifications

  • Review all the codes of the Topic List Watcher and add descriptions of the topic name.
  • Fix the wrong behavior.

Documentation

  • [ ] doc
  • [ ] doc-required
  • [x] doc-not-needed
  • [ ] doc-complete

Matching PR in forked repository

PR in forked repository: x

poorbarcode avatar Feb 05 '24 09:02 poorbarcode