kop icon indicating copy to clipboard operation
kop copied to clipboard

[FEATURE] KOP does not support producer and consumer limits for policies applied to a namespace in pulsar

Open cdhatrak opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe.

Pulsar has following policies applied to namespace. This will specify the number of producers, consumers and subscriptions applied to a topic. But for KOP these limits are not applied

"max_producers_per_topic" "max_consumers_per_topic" "max_consumers_per_subscription" "max_subscriptions_per_topic"

Describe the solution you'd like

MAX Producers Limit: Method registerProducerInPersistentTopic in Class io.streamnative.pulsar.handlers.kop.KafkaTopicManager does not allow multiple topics to be created because of references ConcurrentHashMap.

Hence even if we have limits applied at pulsar side, from kafka side we can create multiple producers exceeding limits applied on pulsar side.

MAX Consumers Limit: If we check the stats for a topic used to consume data in pulsar using command

./pulsar-admin topics stats tenant1/testnm/TOPIC_NAME

In the output of above command we can see that subscriptions are always. Hence the limits for consumers and subscriptions are not working

cdhatrak avatar Aug 03 '22 06:08 cdhatrak