pulsar
pulsar copied to clipboard
support setting bundle number for default namespace when set up cluster
Motivation
In the actual production environment, users may only use one default namespace (public/default). The number of bundles in this namespace is a default value of 16.
- if the cluster has more than 16 brokers, some brokers will serve no traffic
- if the topics in these bundles are not equal, the load will be unbalanced In addition, the manually split bundle is complicated and auto-split may lead to unstable services.
We expect to set a relatively large number of bundles for the default namespace when the cluster is built.
Modifications
support setting bundle number for default namespace when setting up cluster metadata and the bundle number is the defaultNumberOfNamespaceBundles
from broker.conf
Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows: ClusterMetadataSetupTest.testSetupBundleForNamespaces
Documentation
- [x]
doc-not-needed
(Please explain why)
PR in for repository
https://github.com/aloyszhang/pulsar/pull/2
checks in fork repo : https://github.com/aloyszhang/pulsar/pull/2
/pulsarbot run-failure-checks
@merlimat PTAL, thanks
cc @merlimat
@codelipenghui PTAL, thanks
After discussing with @codelipenghui, getting bundle numbers for default namespaces from broker.conf
has a disadvantage in the k8s environment(In k8s, we build pulsar as follows: build zk container, then start a container to init cluster metadata, at this point, we can't access the broker.conf
at all).
So, adding a new option like --default-namespace-bundle-numbers
is better. @merlimat
cc @codelipenghui