[EKS] [request]: Support Availability Zone ID topology labels

[rifelpet]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request Currently AWS CCM populates the topology.kubernetes.io/region and topology.kubernetes.io/zone node labels with the node's region and AZ name respectively:

    topology.kubernetes.io/region: us-east-1
    topology.kubernetes.io/zone: us-east-1a

It would be useful to include a topology label for AZ IDs. topology.kubernetes.io/zone-id or something along those lines. Only the original two labels are standard in the k/k API so we'll need to decide on a new label to use.

    topology.kubernetes.io/zone-id: use1-az1

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

For inter-cluster communication that spans AWS accounts, ensuring zonal isolation becomes challenging when the accounts do not have the same AZ mappings. By including AZ ID information in topologies we can more easily ensure that workloads are scheduled to consistent AZ IDs and communicate with other workloads in the same AZ ID regardless of their AWS account or the account's AZ mapping.

Are you currently working around this issue? No

Additional context https://github.com/kubernetes/cloud-provider-aws/issues/300

Attachments If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[sftim]

topology.kubernetes.io/zone-id is not registered as a Kubernetes label key. To report an AWS-specific detail, I recommend minting one or two new label keys. For example:

• kubernetes.amazonaws.example/availability-zone: us-east-1a
• kubernetes.amazonaws.example/availability-zone-id: use1-az1

[soloradish]

Another use case involves using the AZ's ID for MSK rack-aware consumer settings.

MSK (AWS Managed Kafka) uses AZ's ID as the broker.rack ID, for example, 'broker.rack: apse1-az2'. Making this ID available in node labels can help us inject it into pod environments, which would then allow us to achieve our goal.

[JohnPreston]

Another use case involves using the AZ's ID for MSK rack-aware consumer settings.

MSK (AWS Managed Kafka) uses AZ's ID as the broker.rack ID, for example, 'broker.rack: apse1-az2'. Making this ID available in node labels can help us inject it into pod environments, which would then allow us to achieve our goal.

Have got this exact use-case, with ECS though, but 100%

[JohnPreston]

Another use case involves using the AZ's ID for MSK rack-aware consumer settings.

MSK (AWS Managed Kafka) uses AZ's ID as the broker.rack ID, for example, 'broker.rack: apse1-az2'. Making this ID available in node labels can help us inject it into pod environments, which would then allow us to achieve our goal.

Have got this exact use-case, with ECS though, but 100%

[prmishra]

Did anyone find a workaround for this?

[sbocinec]

The support for the Availability Zone ID labels is now supported by the K8s cloud-provider-aws https://github.com/kubernetes/cloud-provider-aws/pull/855 and available through topology.k8s.aws/zone-id label.

Can someone confirm if the EKS is already using the version of the provider or when it's going to adopt this? Thanks

[dougbyrne]

I see that commit is in the tags v1.30.0-rc.0 and v1.30.0-beta.0. I'm not sure how often something like this would be backported, but we should at least see it in 1.30.

[mikestef9]

This change will be available in upcoming EKS v1.30 release. The feature depends on changes in the core k/k library (v1.30.0) that is used by CCM, and there is not any clear path on backporting to older versions.