Error: infrastructure resource group name is expected to be of format MC_ResourceGroup_ResourceName_Location

[galiacheng]

trafficstars

Describe the bug Got this error when customizing the infrastructure resource group name.

To Reproduce Steps to reproduce the behavior:

1. Deploy AKS with a custom infrastructure resource group name as following:

2. Deploy an App Gateway.

3. Enable ingress controller using command:

az aks enable-addons -n ${AKS_CLUSTER_NAME} -g ${AKS_CLUSTER_RG_NAME} --addons ingress-appgw --appgw-id $appgwId

4. Deploy an App and create ingress

Ingress Controller details

• Output of kubectl describe pod <ingress controller> . The pod name can be obtained by running helm list.

Name:             ingress-appgw-deployment-7556c684c-28pr6
Namespace:        kube-system
Priority:         0
Service Account:  ingress-appgw-sa
Node:             aks-agentpool-37783950-vmss000001/10.224.1.75
Start Time:       Thu, 10 Oct 2024 12:54:24 +0800
Labels:           app=ingress-appgw
                  kubernetes.azure.com/managedby=aks
                  pod-template-hash=7556c684c
Annotations:      checksum/config: 1b53ab68b8204f21dcecbcab8edc3c954d813d6384647ee63538294a8cdba7bd
                  cluster-autoscaler.kubernetes.io/safe-to-evict: true
                  kubernetes.azure.com/metrics-scrape: true
                  prometheus.io/path: /metrics
                  prometheus.io/port: 8123
                  prometheus.io/scrape: true
                  resource-id:
                    /subscriptions/260524c9-7a4d-4483-8d85-de54f9c40ae8/resourceGroups/haiche-aks-rg/providers/Microsoft.ContainerService/managedClusters/haic...
Status:           Running
IP:               10.224.1.147
IPs:
  IP:           10.224.1.147
Controlled By:  ReplicaSet/ingress-appgw-deployment-7556c684c
Containers:
  ingress-appgw-container:
    Container ID:   containerd://beb3267083bfe4712314ab86cdc89cb20606d820f0f0ba8760cde890ce235dc7
    Image:          mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:1.7.4
    Image ID:       mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:92cb29fe2583e4e332f4a6610dd3cd812c2c975d388817934c5c33ff50452a75
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Thu, 10 Oct 2024 12:57:30 +0800
    Last State:     Terminated
      Reason:       Error
      Exit Code:    255
      Started:      Thu, 10 Oct 2024 12:55:16 +0800
      Finished:     Thu, 10 Oct 2024 12:56:47 +0800
    Ready:          True
    Restart Count:  4
    Limits:
      cpu:     700m
      memory:  600Mi
    Requests:
      cpu:      100m
      memory:   20Mi
    Liveness:   http-get http://:8123/health/alive delay=15s timeout=1s period=20s #success=1 #failure=3
    Readiness:  http-get http://:8123/health/ready delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment Variables from:
      ingress-appgw-cm  ConfigMap  Optional: false
    Environment:
      KUBERNETES_SERVICE_HOST:        haicheaks1010-dns-yb0uhxm7.hcp.eastus.azmk8s.io
      KUBERNETES_PORT:                tcp://haicheaks1010-dns-yb0uhxm7.hcp.eastus.azmk8s.io:443
      KUBERNETES_PORT_443_TCP:        tcp://haicheaks1010-dns-yb0uhxm7.hcp.eastus.azmk8s.io:443
      KUBERNETES_PORT_443_TCP_ADDR:   haicheaks1010-dns-yb0uhxm7.hcp.eastus.azmk8s.io
      AZURE_CLOUD_PROVIDER_LOCATION:  /etc/kubernetes/azure.json
      AGIC_POD_NAME:                  ingress-appgw-deployment-7556c684c-28pr6 (v1:metadata.name)
      AGIC_POD_NAMESPACE:             kube-system (v1:metadata.namespace)
      AZURE_ENVIRONMENT:              AZUREPUBLICCLOUD
    Mounts:
      /etc/kubernetes/azure.json from cloud-provider-config (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wkdd7 (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True
  Initialized                 True
  Ready                       True
  ContainersReady             True
  PodScheduled                True
Volumes:
  cloud-provider-config:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/kubernetes/azure.json
    HostPathType:  File
  kube-api-access-wkdd7:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 CriticalAddonsOnly op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

• Output of `kubectl logs .

I1010 05:05:37.219311       1 health_probes.go:55] Created default HTTP probe defaultprobe-Http
I1010 05:05:37.219317       1 health_probes.go:56] Created default HTTPS probe defaultprobe-Http
I1010 05:05:37.219327       1 ingress_rules.go:161] Found backend:sample-domain1-ns/sample-domain1-admin-server
I1010 05:05:37.219336       1 ingress_rules.go:161] Found backend:sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219344       1 ingress_rules.go:161] Found backend:sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219375       1 internaltypes.go:102] Prop name pb-sample-domain1-ns-sample-domain1-cluster-cluster-1-8001-sample-domain1-cluster-appgw-ingress-svc with length 99 is longer than 80 characters; Transformed to pb-sample-domain1-ns-sample-domain1-cluster-clu-50ee70127ff04c1fe0e537071965340b
I1010 05:05:37.219495       1 health_probes.go:70] Created probe pb-sample-domain1-ns-sample-domain1-cluster-clu-50ee70127ff04c1fe0e537071965340b for ingress sample-domain1-ns/sample-domain1-cluster-appgw-ingress-svc at service sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219510       1 internaltypes.go:102] Prop name pb-sample-domain1-ns-sample-domain1-admin-server-7001-sample-domain1-admin-appgw-ingress-svc with length 92 is longer than 80 characters; Transformed to pb-sample-domain1-ns-sample-domain1-admin-serve-971d9872450be68c31408315d764c769
I1010 05:05:37.219576       1 health_probes.go:70] Created probe pb-sample-domain1-ns-sample-domain1-admin-serve-971d9872450be68c31408315d764c769 for ingress sample-domain1-ns/sample-domain1-admin-appgw-ingress-svc at service sample-domain1-ns/sample-domain1-admin-server
I1010 05:05:37.219587       1 internaltypes.go:102] Prop name pb-sample-domain1-ns-sample-domain1-cluster-cluster-1-8001-sample-domain1-cluster-appgw-ingress-https-svc with length 105 is longer than 80 characters; Transformed to pb-sample-domain1-ns-sample-domain1-cluster-clu-f28d9e51906815d92111eafe019a24d8
I1010 05:05:37.219649       1 health_probes.go:70] Created probe pb-sample-domain1-ns-sample-domain1-cluster-clu-f28d9e51906815d92111eafe019a24d8 for ingress sample-domain1-ns/sample-domain1-cluster-appgw-ingress-https-svc at service sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219667       1 internaltypes.go:102] Prop name bp-sample-domain1-ns-sample-domain1-cluster-cluster-1-8001-8001-sample-domain1-cluster-appgw-ingress-svc with length 104 is longer than 80 characters; Transformed to bp-sample-domain1-ns-sample-domain1-cluster-clu-5acd5d6e7ecdd2c0e0283deab6b20722
I1010 05:05:37.219684       1 backendhttpsettings.go:89] Created backend http settings bp-sample-domain1-ns-sample-domain1-cluster-clu-5acd5d6e7ecdd2c0e0283deab6b20722 for ingress sample-domain1-ns/sample-domain1-cluster-appgw-ingress-svc and service sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219705       1 internaltypes.go:102] Prop name bp-sample-domain1-ns-sample-domain1-admin-server-7001-7001-sample-domain1-admin-appgw-ingress-svc with length 97 is longer than 80 characters; Transformed to bp-sample-domain1-ns-sample-domain1-admin-serve-e41563c43fe0154a4193a052d1c2fbc9
I1010 05:05:37.219718       1 backendhttpsettings.go:89] Created backend http settings bp-sample-domain1-ns-sample-domain1-admin-serve-e41563c43fe0154a4193a052d1c2fbc9 for ingress sample-domain1-ns/sample-domain1-admin-appgw-ingress-svc and service sample-domain1-ns/sample-domain1-admin-server
I1010 05:05:37.219729       1 internaltypes.go:102] Prop name bp-sample-domain1-ns-sample-domain1-cluster-cluster-1-8001-8001-sample-domain1-cluster-appgw-ingress-https-svc with length 110 is longer than 80 characters; Transformed to bp-sample-domain1-ns-sample-domain1-cluster-clu-ec9620488456fe498e4a8d167834f6dd
I1010 05:05:37.219741       1 backendhttpsettings.go:89] Created backend http settings bp-sample-domain1-ns-sample-domain1-cluster-clu-ec9620488456fe498e4a8d167834f6dd for ingress sample-domain1-ns/sample-domain1-cluster-appgw-ingress-https-svc and service sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219752       1 backendaddresspools.go:36] Created default backend pool defaultaddresspool
I1010 05:05:37.219764       1 backendaddresspools.go:47] Created backend pool pool-sample-domain1-ns-sample-domain1-cluster-cluster-1-8001-bp-8001 for service sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219775       1 backendaddresspools.go:47] Created backend pool pool-sample-domain1-ns-sample-domain1-admin-server-7001-bp-7001 for service sample-domain1-ns/sample-domain1-admin-server
I1010 05:05:37.219781       1 backendaddresspools.go:47] Created backend pool pool-sample-domain1-ns-sample-domain1-cluster-cluster-1-8001-bp-8001 for service sample-domain1-ns/sample-domain1-cluster-cluster-1
I1010 05:05:37.219791       1 frontend_listeners.go:116] Processing Rules for Ingress: sample-domain1-ns/sample-domain1-admin-appgw-ingress-svc
I1010 05:05:37.219803       1 configbuilder.go:238] Derived listener port from ingress: 80
I1010 05:05:37.219811       1 frontend_listeners.go:116] Processing Rules for Ingress: sample-domain1-ns/sample-domain1-cluster-appgw-ingress-https-svc
I1010 05:05:37.219818       1 ingress_rules.go:81] Found annotation appgw-ssl-certificate: appGatewaySslCert in ingress sample-domain1-ns/sample-domain1-cluster-appgw-ingress-https-svc
I1010 05:05:37.219823       1 configbuilder.go:238] Derived listener port from ingress: 443
I1010 05:05:37.219830       1 frontend_listeners.go:116] Processing Rules for Ingress: sample-domain1-ns/sample-domain1-cluster-appgw-ingress-svc
I1010 05:05:37.219839       1 configbuilder.go:238] Derived listener port from ingress: 80
I1010 05:05:37.219867       1 configbuilder.go:238] Derived listener port from ingress: 80
I1010 05:05:37.219917       1 requestroutingrules.go:410] Attached pool pool-sample-domain1-ns-sample-domain1-admin-server-7001-bp-7001 and http setting bp-sample-domain1-ns-sample-domain1-admin-serve-e41563c43fe0154a4193a052d1c2fbc9 to path rule: pr-sample-domain1-ns-sample-domain1-admin-appgw-ingress-svc-rule-0-path-0
I1010 05:05:37.219927       1 ingress_rules.go:81] Found annotation appgw-ssl-certificate: appGatewaySslCert in ingress sample-domain1-ns/sample-domain1-cluster-appgw-ingress-https-svc
I1010 05:05:37.219932       1 configbuilder.go:238] Derived listener port from ingress: 443
I1010 05:05:37.219977       1 configbuilder.go:238] Derived listener port from ingress: 80
I1010 05:05:37.220019       1 requestroutingrules.go:472] Getting Request Routing Rules Priority for Ingress: sample-domain1-ns/sample-domain1-admin-appgw-ingress-svc
I1010 05:05:37.220028       1 configbuilder.go:238] Derived listener port from ingress: 80
I1010 05:05:37.220037       1 requestroutingrules.go:472] Getting Request Routing Rules Priority for Ingress: sample-domain1-ns/sample-domain1-cluster-appgw-ingress-https-svc
I1010 05:05:37.220042       1 ingress_rules.go:81] Found annotation appgw-ssl-certificate: appGatewaySslCert in ingress sample-domain1-ns/sample-domain1-cluster-appgw-ingress-https-svc
I1010 05:05:37.220047       1 configbuilder.go:238] Derived listener port from ingress: 443
I1010 05:05:37.220054       1 requestroutingrules.go:472] Getting Request Routing Rules Priority for Ingress: sample-domain1-ns/sample-domain1-cluster-appgw-ingress-svc
I1010 05:05:37.220063       1 configbuilder.go:238] Derived listener port from ingress: 80
I1010 05:05:37.220080       1 requestroutingrules.go:124] Bound basic rule: rr-1c526ee8ba68b3db940684e9093980ad to listener: fl-1c526ee8ba68b3db940684e9093980ad ([    ], 443) for backend pool pool-sample-domain1-ns-sample-domain1-cluster-cluster-1-8001-bp-8001 and backend http settings bp-sample-domain1-ns-sample-domain1-cluster-clu-ec9620488456fe498e4a8d167834f6dd
I1010 05:05:37.220095       1 requestroutingrules.go:119] Bound path-based rule: rr-452c578b4f742bd7a3927c3caf2b604e to listener: fl-452c578b4f742bd7a3927c3caf2b604e ([    ], 80) and url path map url-452c578b4f742bd7a3927c3caf2b604e
I1010 05:05:37.228863       1 azure.go:87] Code="ErrorMissingResourceGroup" Message="infrastructure resource group name: haiche-aks-node-rg is expected to be of format MC_ResourceGroup_ResourceName_Location"
I1010 05:05:37.228874       1 configbuilder.go:282] Error while parsing cluster resource ID for tagging: Code="ErrorMissingResourceGroup" Message="infrastructure resource group name: haiche-aks-node-rg is expected to be of format MC_ResourceGroup_ResourceName_Location"
I1010 05:05:37.229529       1 mutate_app_gateway.go:153] cache: Config has NOT changed! No need to connect to ARM.
I1010 05:05:37.229539       1 controller.go:152] Completed last event loop run in: 74.089474ms

• Any Azure support tickets associated with this issue. No