cp-helm-charts
cp-helm-charts copied to clipboard
confluentinc
Define broker.rack propery
Kubernetes knows the availability zone or rack a pod is running on.
We could take this info, which is in the annotation of the node called failure-domain.beta.kubernetes.io/zone, and set it as the broker.rack property.
AFAIK, That annotation is setup by kube cloud integration's automatically, i.e. on AWS it takes values like us-east-1c.
On-Prem, that annotation can be manually set, but not enforced.
In this case the property would be set as:
broker.rack=
Not exactly the same as the default null value, but close enough as both makes kafka behave the same 😅
Thanks!!
It looks like @killfill hasn't signed our Contributor License Agreement, yet.
The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence. Wikipedia
You can read and sign our full Contributor License Agreement here.
Once you've signed reply with [clabot:check] to prove it.
Appreciation of efforts,
clabot
It looks like @killfill hasn't signed our Contributor License Agreement, yet.
The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence. Wikipedia
You can read and sign our full Contributor License Agreement here.
Once you've signed reply with [clabot:check] to prove it.
Appreciation of efforts,
clabot
@confluentinc It looks like @killfill just signed our Contributor License Agreement. :+1:
Always at your service,
clabot
@gAmUssA You mean something like that?
I would guess it's a good idea to leave the rack awareness enabled by default. When k8s is on cloud, it actually does has zones. When on-prem, its 'optional'. If disabled, the rack property will be empty (disabled)
Its like the user is defining it its infra has multiple 'racks' (or zones) on k8s, and maybe kafka should just inherit it (?).
Thanks!
I'd also recommend making the topology key configurable. For example, we're deploying our kafka worker nodes into a PARTITION placement group so the partition number should also be included in the rack id. Unfortunately, since these placement groups are new, the AWS cloud provider doesn't add it to the zone annotation. It would be useful to be able to specify the annotation to use as the key. Then we could annotate our nodes with a custom annotation that contains both the AZ and the partition id.
hi @gAmUssA - If we add above changes confluent Kafka cluster will deploy on availability zones on EKS Cluster, using Rack Awareness. Which parameter I need to pass for broke.rack= in values.yaml file.
I have deployed with above changes, I am getting below error. Please suggest.
Thanks.
