[EKS] [kube-proxy addon]: Adding custom label and annotations on kube-proxy Addon

[avnerv]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request Adding custom annotations like cloud.xxx.ai/owner-display-name: infra on kube-proxy addon so that it can be put on pods.

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? It seems like there is potentially a missing setting on the DaemonSet, the custom annotation via the configuration_values. The example terraform configuration:

resource "aws_eks_addon" "kube_proxy" {
  cluster_name      = aws_eks_cluster.eks.id
  addon_name        = "kube-proxy"
  addon_version     = "v1.25.11-eksbuild.2"
  resolve_conflicts = "OVERWRITE"
  configuration_values = jsonencode({
    podAnnotations = {
      "cloud.xxx.ai/owner-display-name" = "infra"
    }
  })
}

The terraform error message:

│ Error: error updating EKS Add-On (hadc-xxxxx-eks:kube-proxy): InvalidParameterException: ConfigurationValue provided in request is not supported: Json schema validation failed with error: [$.podAnnotations: is not defined in the schema and the schema does not allow additional properties]
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "1c971b0c-bb71-400e-b8dc-f4b64df2180c"
│   },
│   AddonName: "kube-proxy",
│   ClusterName: "hadc-xxxxx-eks",
│   Message_: "ConfigurationValue provided in request is not supported: Json schema validation failed with error: [$.podAnnotations: is not defined in the schema and the schema does not allow additional properties]"
│ }
│ 
│   with module.eks_cluster.aws_eks_addon.kube_proxy,
│   on ../../modules/aws/eks/cluster/addon.tf line 14, in resource "aws_eks_addon" "kube_proxy":
│   14: resource "aws_eks_addon" "kube_proxy" {
│ 
╵
Releasing state lock. This may take a few moments...

Are you currently working around this issue? How are you currently solving this problem?

Additional context Anything else we should know?

Attachments If you think you might have additional information that you'd like to include via an attachment, please do - we'll take a look. (Remember to remove any personally-identifiable information.)

[Raj-Popat]

@mikestef9 Any updates on this. Do we have any timeline when the proposed change would get added?

[ri-roee]

Wanted to follow up on this and say that this did work for me, might be a stale issue

[avnerv]

Wanted to follow up on this and say that this did work for me, might be a stale issue

for what kube-proxy addon version?

[ri-roee]

v1.16.2-eksbuild.1 on EKS 1.25

Config (used for DD scrape of prometheus-compatible metrics endpoint):

resource "aws_eks_addon" "vpc_cni_addon" {
  cluster_name      = module.eks.cluster_id
  addon_name        = "vpc-cni"
  addon_version     = data.aws_eks_addon_version.vpc_cni_latest.version
  resolve_conflicts = "OVERWRITE"

  configuration_values = jsonencode({
    podAnnotations = {
      "prometheus.io/scrape" = "true"
    }
  })
}

[avnerv]

v1.16.2-eksbuild.1 on EKS 1.25 Config (used for DD scrape of prometheus-compatible metrics endpoint):

sorry but not sure i follow, the example you provided above is for vpc-cni, whereas the issue we're discussing pertains to kube-proxy

[ri-roee]

My bad, copied the wrong config. Same concept applies to the kube-proxy addon

[avnerv]

My bad, copied the wrong config. Same concept applies to the kube-proxy addon

for which version? in v1.27.8-eksbuild.4 is not supported yet

│ Error: error updating EKS Add-On (test-eks:kube-proxy): InvalidParameterException: ConfigurationValue provided in request is not supported: Json schema validation failed with error: [$.podAnnotations: is not defined in the schema and the schema does not allow additional properties]
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "fb118817-2124-4c1a-b9a2-d1d93c1abeb7"
│   },
│   AddonName: "kube-proxy",
│   ClusterName: "test-eks",
│   Message_: "ConfigurationValue provided in request is not supported: Json schema validation failed with error: [$.podAnnotations: is not defined in the schema and the schema does not allow additional properties]"
│ }

[sjastis]

This is now supported with EKS kube-proxy add-on version, starting with Kube-proxy v1.25.16-eksbuild.3, v1.26.13-eksbuild.2, v1.27.10-eksbuild.2, v1.28.6-eksbuild.2, v1.29.1-eksbuild.2 Doc Updated: (https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html)