pulsar
pulsar copied to clipboard
[fix] [broker] Fix can not subscribe partitioned topic with a suffix-matched regexp
Motivation
If a consumer tries to subscribe to a partitioned topic with a suffix-matched regexp, it does not work.
- create topic
persistent://public/default/tp - create a consumer with regexp
persistent://public/default/tp$ - the pattern consumer has no internal consumers.
You can reproduce the issue by the test testPreciseRegexpSubscribe.
- it can work when using
2.10.x - it can not work when using
>=2.11.0, it is a break change due to PIP-145
https://github.com/apache/pulsar/pull/21885 fixed one wrong logic here, but there is still another wrong logic, see https://github.com/apache/pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/TopicListService.java#L64
@Override
public void accept(String topicName, NotificationType notificationType) {
// The param "topicName" contains the partition suffix, which causes a suffix-matched regexp not to match.
if (topicsPattern.matcher(topicName).matches()) {
List<String> newTopics;
List<String> deletedTopics;
...
Why can the test testPreciseRegexpSubscribe pass?
https://github.com/apache/pulsar/pull/21885 added a test testPreciseRegexpSubscribe that covers the test which creates a consumer with a suffix-matched regexp. It works as below:
- create a consumer with a suffix-matched regexp
- create a partitioned topic
- two events will trigger the pattern consumer to auto-create an internal single partition consumer.
- Start a new
recheckTopicsChangeAfterReconnectafter the topic watcher is connected. https://github.com/apache/pulsar/pull/21885 fixed the wrong behavior. - Receive an event that a new topic created. Current PR is trying to fix the wrong behavior.
- Start a new
The test testPreciseRegexpSubscribe can not pass as the below flow:
- create a consumer with a suffix-matched regexp
- wait for Topic List Watcher to start.
- the consumer will call
recheckTopicsChangeAfterReconnectat this step.
- the consumer will call
- create a partitioned topic
- the consumer can not receive the event due to the wrong logic, then the test failed.
Modifications
- Review all the codes of the Topic List Watcher and add descriptions of the
topic name. - Fix the wrong behavior.
Documentation
- [ ]
doc - [ ]
doc-required - [x]
doc-not-needed - [ ]
doc-complete
Matching PR in forked repository
PR in forked repository: x