[Bug] Partition topic cannot be matched using regex

[hecter-java-architect]

Search before asking

• [X] I searched in the issues and found nothing similar.

Version

pulsarVersion:3.0.0 pulsarClientVersion:3.0.0

Minimal reproduce step

When the server receives a request of type GET_TOPICS_OF_NAMESPACE, it will return the Topic and perform regular matching However, the returned Topic contains the "-partition-0" partition suffix, but when matching again, the partition suffix is ​​not removed

getBrokerService().pulsar().getNamespaceService().getListOfTopics(namespaceName, mode)
                        .thenAccept(topics -> {
                            boolean filterTopics = false;
                            // filter system topic
                            List<String> filteredTopics = TopicList.filterSystemTopic(topics);
                            if (enableSubscriptionPatternEvaluation && topicsPattern.isPresent()) {
                                if (topicsPattern.get().length() <= maxSubscriptionPatternLength) {
                                    filterTopics = true;
                                    filteredTopics = TopicList.filterTopics(filteredTopics, topicsPattern.get());
                                } else {
                                    log.info("[{}] Subscription pattern provided was longer than maximum {}.",
                                            remoteAddress, maxSubscriptionPatternLength);
                                }
                            }
}

Let's take a look filterSystemTopic

List<String> filteredTopics = TopicList.filterSystemTopic(topics);

public static List<String> filterSystemTopic(List<String> original) {
        return original.stream()
                .filter(topic -> !SystemTopicNames.isSystemTopic(TopicName.get(topic)))
                .collect(Collectors.toList());
    }
public static boolean isSystemTopic(TopicName topicName) {
        TopicName nonPartitionedTopicName = TopicName.get(topicName.getPartitionedTopicName());
        return isEventSystemTopic(nonPartitionedTopicName) || isTransactionInternalName(nonPartitionedTopicName);
    }

What did you expect to see?

Hope that the matching topic can be removed from the partition suffix

What did you see instead?

The matching topic does not remove the partition suffix

Anything else?

No response

Are you willing to submit a PR?

• [X] I'm willing to submit a PR!

[hecter-java-architect]

test

public static void main(String[] args) {
        List<String> topics = Lists.newArrayList(
                "persistent://local-public/default/mixed-load-1-0-partition-0",
                "persistent://local-public/default/mixed-load-1-1-partition-0",
                "persistent://local-public/default/mixed-load-no-partten-1-0-partition-0");
        List<String> filteredTopics = TopicList.filterSystemTopic(topics);

        filteredTopics = TopicList.filterTopics(filteredTopics, "local-public/default/mixed-load-1-\\d{1,3}");
        System.out.println(filteredTopics);
    }

result

[]

Process finished with exit code 0

[hecter-java-architect]

The data of the /managed-ledgers path under zookeeper is very clear, so it will not be displayed

[hecter-java-architect]

I found that in the filterTopics method, there are comments showing the section that should not contain partitions

// get topics that match 'topicsPattern' from original topics list
    // return result should contain only topic names, without partition part
    public static List<String> filterTopics(List<String> original, String regex) {
        Pattern topicsPattern = Pattern.compile(regex);
        return filterTopics(original, topicsPattern);
    }
    public static List<String> filterTopics(List<String> original, Pattern topicsPattern) {

        final Pattern shortenedTopicsPattern = topicsPattern.toString().contains(SCHEME_SEPARATOR)
                ? Pattern.compile(SCHEME_SEPARATOR_PATTERN.split(topicsPattern.toString())[1]) : topicsPattern;

        return original.stream()
                .map(TopicName::get)
                .map(TopicName::toString)
                .filter(topic -> shortenedTopicsPattern.matcher(SCHEME_SEPARATOR_PATTERN.split(topic)[1]).matches())
                .collect(Collectors.toList());
    }

[hecter-java-architect]

return original.stream()
                .map(TopicName::get)
// should use  TopicName::getPartitionedTopicName
                .map(TopicName::toString)
                .filter(topic -> shortenedTopicsPattern.matcher(SCHEME_SEPARATOR_PATTERN.split(topic)[1]).matches())
                .collect(Collectors.toList());

[mattisonchao]

Hi, @hecter-java-architect

You can check the discussion here to get the workaround fix. https://github.com/apache/pulsar/discussions/20716#discussioncomment-6679380

I'll start discussing it to see what behaviour is correct, and then we can let them reach a consensus.

[hecter-java-architect]

你好，@hecter-java-architect

您可以查看此处的讨论以获取解决方法。#20716（在帖子中回复）

我会开始讨论，看看什么行为是正确的，我们就可以让他们达成共识然后。

Thanks，I think that after this configuration enableBrokerSideSubscriptionPatternEvaluation is false, all topics will be returned to the client for filtering, which is not a good practice, And the filterTopics method also hopes not to include partition part，So I want to know why not remove the partition information on the server side for filtering。

[mattisonchao]

@hecter-java-architect

Yep, You are right. We should consider changing it, but it was introduced by a new proposal. let me issue a discussion for it.

[hecter-java-architect]

@hecter-java-architect

Yep, You are right. We should consider changing it, but it was introduced by a new proposal. let me issue a discussion for it.

sure，when we can start discussing this proposal，i want to join

[github-actions[bot]]

The issue had no activity for 30 days, mark with Stale label.