application-gateway-kubernetes-ingress
application-gateway-kubernetes-ingress copied to clipboard
AG ingress ignores changes to default backend
Describe the bug
Application gateway ignores k8s events related to defaultBackend
pods/endpoints.
As a result, it doesn't update the backend pool target address on pod recreation.
To Reproduce
- Enable Azure ingress in the cluster
- Apply the following yaml:
apiVersion: apps/v1 kind: Deployment metadata: name: my-nginx labels: app.kubernetes.io/name: "my-nginx" spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: "my-nginx" template: metadata: labels: app.kubernetes.io/name: "my-nginx" spec: containers: - name: nginx image: nginx:1.20 ports: - containerPort: 80 name: http --- apiVersion: apps/v1 kind: Deployment metadata: name: default-nginx labels: app.kubernetes.io/name: "default-nginx" spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: "default-nginx" template: metadata: labels: app.kubernetes.io/name: "default-nginx" spec: containers: - name: nginx image: nginx:1.21 ports: - containerPort: 80 name: http --- apiVersion: v1 kind: Service metadata: name: default-service spec: ports: - name: http protocol: TCP port: 80 targetPort: http selector: app.kubernetes.io/name: "default-nginx" type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: my-service spec: ports: - name: http protocol: TCP port: 8080 targetPort: http selector: app.kubernetes.io/name: "my-nginx" type: ClusterIP --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: test-ingress annotations: kubernetes.io/ingress.class: azure/application-gateway appgw.ingress.kubernetes.io/backend-path-prefix: "/" spec: defaultBackend: service: name: default-service port: number: 80 rules: - http: paths: - path: /test-my-ingress/* pathType: Prefix backend: service: name: my-service port: number: 8080
- Check that both services work
- Restart
default-nginx
:kubectl rollout restart deployment/default-nginx
- Wait 30 seconds
- Make sure that default backend is not accessible and AG returns 502/504
Ingress Controller details
How to fix
Add
for _, ingress := range c.ListHTTPIngresses() {
defaultBackend := ingress.Spec.DefaultBackend
if defaultBackend != nil && (defaultBackend.Service.Name == service.Name) {
return true
}
// ...
}
to https://github.com/Azure/application-gateway-kubernetes-ingress/blob/master/pkg/k8scontext/context.go#L850-L866
At the moment it only cares about rules
.