Two AGICs watching same namespace ignoring ingressClassName

[melzayet]

Describe the bug I have two application gateways controlled by two AGICs 1.5.2 deployed through Helm. Two ingress classes are created successfully. Yet when creating two ingress objects in same namespace and configuring each with its corresponding ingressClassName, both Application Gateways configure all rules in all ingresses

Ideally, ingressClassName should limit ingress to be handled by only matching IngressClass

To Reproduce Steps to reproduce the behavior:

1. Create two separate App Gateways
2. Deploy two AGICs (AGIC1, AGIC2) through Helm with default configuration and specifying unique "kubernetes.ingressClassResource.name"
3. Create ingress1 in namespace NS with ingressClassName set to AGIC1

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  creationTimestamp: "2022-07-20T17:24:53Z"
  name: agic-backend-caller
  namespace: apps
spec:
  ingressClassName: my-custom-ingress
......

5. Create ingress2 in same namespace NS with ingressClassName set to AGIC2

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: agic-php-apache
  namespace: apps
spec:
  ingressClassName: my-custom-ingress-2

7. Check App Gateways listeners and rules, both ingress1 and ingress2 rules are applied to both gateways

Ingress Controller details

• Output of kubectl describe pod <ingress controller> . The pod name can be obtained by running helm list.

Name:         ingress-azure-1658337849-59dc7c989-ffktz
Namespace:    apps
Priority:     0
Node:         aks-agentpool-26601273-vmss000001/172.20.8.116
Start Time:   Wed, 20 Jul 2022 19:24:12 +0200
Labels:       aadpodidbinding=ingress-azure-1658337849
              app=ingress-azure
              pod-template-hash=59dc7c989
              release=ingress-azure-1658337849
Annotations:  checksum/config: 37bd7de72fdec1e5dbcf63bc2bca6de6e4e2ed12bde25ad2a0f835e53da18184
              prometheus.io/port: 8123
              prometheus.io/scrape: true
Status:       Running
IP:           172.20.8.172
IPs:
  IP:           172.20.8.172
Controlled By:  ReplicaSet/ingress-azure-1658337849-59dc7c989
Containers:
  ingress-azure:
    Container ID:   containerd://4bb413c7be0abc642af0a06b3f89e9ac3c056e58ac4d40660cd26ba3d2f47107
    Image:          mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.5.2
    Image ID:       mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:69a8f8ea51e71e67041323668ca3b250f4316147d8872c26e6bd12d032b2fa06
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Wed, 20 Jul 2022 19:24:13 +0200
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:8123/health/alive delay=15s timeout=1s period=20s #success=1 #failure=3
    Readiness:      http-get http://:8123/health/ready delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      ingress-azure-1658337849  ConfigMap  Optional: false
    Environment:
      AZURE_CLOUD_PROVIDER_LOCATION:  /etc/appgw/azure.json
      AGIC_POD_NAME:                  ingress-azure-1658337849-59dc7c989-ffktz (v1:metadata.name)
      AGIC_POD_NAMESPACE:             apps (v1:metadata.namespace)
    Mounts:
      /etc/appgw/ from azure (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-92pp5 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  azure:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/kubernetes/
    HostPathType:  Directory
  kube-api-access-92pp5:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

• Output of `kubectl logs .

I0725 07:22:04.901712       1 mutate_app_gateway.go:164] cache: Config has NOT changed! No need to connect to ARM.
I0725 07:22:04.901735       1 controller.go:151] Completed last event loop run in: 195.109458ms
I0725 07:22:06.074416       1 request.go:600] Waited for 91.345944ms due to client-side throttling, not priority and fairness, request: PUT:https://10.0.0.1:443/apis/networking.k8s.io/v1/namespaces/development/ingresses/agic-backend-caller/status
I0725 07:22:06.091028       1 backendhttpsettings.go:89] Created backend http settings bp-apps-php-apache-80-80-agic-php-apache for ingress apps/agic-php-apache and service apps/php-apache
I0725 07:22:06.091058       1 backendhttpsettings.go:89] Created backend http settings bp-default-kubernetes-443-443-agic-php-apache for ingress default/agic-php-apache and service default/kubernetes
I0725 07:22:06.091089       1 backendhttpsettings.go:89] Created backend http settings bp-development-backend-caller-80-8000-agic-backend-caller for ingress development/agic-backend-caller and service development/backend-caller

• Any Azure support tickets associated with this issue. 2207130030002317

[akshaysngupta]

@melzayet Thanks for pointing out. This looks like a bug in AGIC in the following line: https://github.com/Azure/application-gateway-kubernetes-ingress/blob/b875ca5514f335930650db5099a4f8c74fc8cfaa/pkg/k8scontext/context.go#L944 It also needs to compare the controller's name along with the controller type. We will fix this at our earliest.

[tbertenshaw]

Hi @akshaysngupta, I can see this issue still open despite a merged PR. Is it resolved? We are considering upgrading and this will impact us.

[PeterThomasAwen]

This is also having an impact on us, We are trying to move from nginx to AGIC and this js a significant blocker to use using your service.