containers-roadmap
containers-roadmap copied to clipboard
aws
[EKS]: EKS Cluster Tagging Propagation
Tell us about your request Ability to propogate EKS cluster tags to EC2 resources (instances, autoscaling groups, load balancers) used as part of the cluster. This will help with resource organization and cost allocation within your AWS account.
Feedback requested Should we expand the scope of this to include propagating K8s labels to AWS tags and giving you the ability to assign labels used for resources within certain namespaces, services, etc as AWS tags on related resources? This is one path to improving cost allocation capabilities withing AWS and K8s.
Should we expand the scope of this to include propagating K8s labels to AWS tags and giving you the ability to assign labels used for resources within certain namespaces, services, etc as AWS tags on related resources?
Yes - I think this is a great idea. It would be a really ace integration and, other than just for cost allocation, it would make it easier to programatically select k8s-managed resources in custom ops scripting.
First, make it work. Then make it work better. Expanded scope is great as long as it does not delay the release date.
I was just deploying a ALB using ALB Ingress controller and I notice that we do have the ability to tag it by annotating the Ingress.
annotations: alb.ingress.kubernetes.io/tags: auto-delete=no,owner=gmagella,app=my-1st-flask
Looks like it does tag the ALB and the TG. But I couldn't help but notice that it didn't update tags on the ALB, when changing the Ingress details. (it does update the TG tags though).
Let me know if this is not the right venue to comment on this, and I'll take appropriate actions.
Did the tags used to work? I thought they did but all my nodeGroups now no longer have the tags specified in the eksctl configuration. Need to get these tags back in as they are used for Cost reporting.
[edit] eksctl can propagate tags for self-managed and managed node groups. however if you use CFN or the API, tags for EKS managed node groups do not propagate. We are working on enabling tag propagation for EKS clusters from the cluster down to nodes through the EKS API.
I'm not exactly sure which issue to put this on so I'll probably but it in a decent number of issues but not having propagating Tags to all EKS cluster resources effectively makes this entire service useless. If you can't tag your resources you can't separate out costs between different programs and therefore will never be allowed by a program manager and rightfully so. This needs to be fixed ASAP.
it would be great if labels on EKS resources could be used as cost-allocation tags. The few taggable EKS resources currently do minimal to help us with cost allocation. We need tagging and propagation.
I agree with the above statement around making it work as the priority. With the heavy dependency and enforcement around tagging, it is upsetting to see that the behavior we have come to expect with tag propagation is not present in EKS. This is causing us to have to create exceptions and workarounds to problems such as auto scaling groups not being tagged and slowing down velocity as we look to migrate from ECS to EKS.
It would be great to understand what sort of time line we are looking at for resolving this as the priority.
Outside of fixing what is missing today, enabling additional k8s label propagation to aid in further dissecting costs is secondary, I would think. For us, fixing the tag propagation actually gives all of the data we need for cost allocation in EKS.
eksctl 0.48 will be supporting user defined tagging for EKS resources. Please see https://github.com/weaveworks/eksctl/releases/tag/0.48.0-rc.0
Tags will be propagated to managed and self-managed nodes and its resources such as instances and ebs volumes
+1. It'll be very good to propagate user defined tags to ENIs using Launch Template
Any update on this? Tag propagation from eks node groups to ec2 instances is a no brainer. Some of us use CDK to define our infrastructure and now there is no way to add any custom tags to EKS managed nodes.
For instances specifically, you can set the tags in the Launch Template instead of setting them somewhere higher and propagating them down.
As well as being the approach that AWS has already implemented and recommends for this use-case, this mechanism has the advantage that you can propagate those or other tags to associated EBS volumes for the instance, which ASG-propagated tags do not support. See Tag Auto Scaling groups and instances and Creating a launch template for an Auto Scaling group for reference.
I don't think eksctl supports any propagation beyond what EKS already supports. It does have some features to copy tags around from place to place, mostly related to what I described in my previous comment, and for working around #608.
That doesn't cover the use cases described in this ticket, such as "Setting a tag CostCentre=Project2 on the EKS Cluster object will automatically put that tag on the NLB instance created for a LoadBalancer Service".
@TBBle eksctl do support tag propagation since version 0.48. However, tags are only propagated to ec2 and ebs volumes.
ALB or ELB are not created when EKS cluster is created. However, you can always use annotations to push tags to load balancers. see https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/service/annotations/
Disclaimer: I am contributor for tagging support in EKS (https://github.com/weaveworks/eksctl/pull/3639)
That was my point. The only tag propagation eksctl supports is the one EKS implements: from ASGs to their EC2 instances and from Launch Templates to their EC2 instances and EBS instances. That's the PR you linked to, which implements the eksctl side of that EKS feature. It doesn't do anything more for this than can be done with e.g., CDK or terraform, because it calls the same APIs.
This ticket is asking for exactly the propagation you've noted, for things that do not exist when the cluster is created, but are created later, and still need to be visible for, e.g. cost allocation. Because eksctl is not running "later", it can't support that without an EKS feature behind it. Which is what this ticket is effectively requesting.
The workaround for NLB for example is annotations on the Load Balancer, but that's not propagation, that's just repeating the same tags all through your config (and other cluster user's configs).
I just encountered another location that tag propagation would be helpful. Granting access via Tags in AWS CloudWatch. the /aws/eks/*/clusters:* audit logs don't have any tags attached from the eks cluster.
+1. It'd be useful especially we are working from Terraform environment.
While EKS does not support custom tag propagation which this issue covers, a noteworthy recent improvement we have made is adding a cluster name AWS cost allocation tag to every EC2 instance that joins an EKS cluster. Details in the What's New post.
What we are really expecting is to be able to work with our own tagging system already in place.
[edit] eksctl can propagate tags for self-managed and managed node groups. however if you use CFN or the API, tags for EKS managed node groups do not propagate. We are working on enabling tag propagation for EKS clusters from the cluster down to nodes through the EKS API.
@tabern 2 years have passed, still working on it, or should this issue be moved to another column on the roadmap?
IMHO the "use eksctl" approach is not a valid one since if you're doing things in a repeatable way by using terraform, or the likes of it (anything that relies on the API) you won't be using eksctl.
This issue messes up our billing breakdown pretty severely since it relies on tags and therefore there is a lot of unaccounted for resources in the end of the month.
For anyone using terraform, this comment is gold: https://github.com/aws/containers-roadmap/issues/781#issuecomment-1443961282 Tagging the autoscaling group only covers one aspect of the original request but load balancers can be handled with annotations if you're using the load balancer controller.
We would like to see this implemented as well, specifically for the Cloud Watch Log Groups
