aws-load-balancer-controller icon indicating copy to clipboard operation
aws-load-balancer-controller copied to clipboard

Readiness Gate Injection Breaks Kubernetes 1.29 Sidecar Pods

Open jlrgraham23 opened this issue 10 months ago • 0 comments

Describe the bug

When the pod readiness gate feature is enabled on a namespace in a Kubernetes 1.29 cluster it strips away the restartPolicy: Always values on containers in the initContainers block; this effectively breaks the new SideCar Containers feature in 1.29.

This looks to be due to use of an older version of the k8s.io/api library here: https://github.com/kubernetes-sigs/aws-load-balancer-controller/blob/main/go.mod#L22C2-L22C20

Same issue here with the EKS specific Pod Identity Webhook.

  • Issue: https://github.com/aws/amazon-eks-pod-identity-webhook/issues/208
  • PR: https://github.com/aws/amazon-eks-pod-identity-webhook/pull/209

Upstream docs: https://kubernetes.io/blog/2023/08/25/native-sidecar-containers/

Steps to reproduce

  • Startup a Kubernetes 1.29 cluster.
  • Create a namespace with the elbv2.k8s.aws/pod-readiness-gate-inject=enabled label.
  • Deploy a workload into this namespace using restartPolicy: Always on an init container in a pod.
  • Receive infinite hang as the sidecar containers special status is removed and the pod tries to wait on them for startup.
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: foobar
spec:
  replicas: 2
  selector:
    matchLabels:
      app: foobar
  template:
    metadata:
      labels:
        app: foobar
    spec:
      initContainers:
        - name: sidecar
          image: debian
          command: ["sleep", "60"]
          restartPolicy: Always
      containers:
        - name: main
          image: debian
          command: ["sleep", "60"]

Inspecting the resulting pods from this Deployment shows that the restartPolicy attribute has been removed in flight.

Expected outcome

The readiness gate should not remove the restartPolicy attribute from containers in the initContainers spec.

Environment

  • AWS Load Balancer controller version: v2.7.2
  • Kubernetes version: 1.29.1
  • Using EKS (yes/no), if so version? Yes, 1.29.

jlrgraham23 avatar Apr 16 '24 19:04 jlrgraham23