aks-app-routing-operator icon indicating copy to clipboard operation
aks-app-routing-operator copied to clipboard

[BUG] `keyvault-nginx-default` is stuck in ContainerCreating

Open JoeyC-Dev opened this issue 7 months ago • 2 comments

Problem: Cannot set default certificate via Key Vault.

Screenshot: image image image

Caption: I am not sure if I configured permission correctly, but the naming of "placeholder" and the use of "pause" image makes me thinking if this is intended. There is no other log I can find.

Set-up demo:

ranNum=$(echo $RANDOM)
region=westus
rG=aks_approuting_${ranNum}
kv=kv${ranNum}
aks=aks-${ranNum}
aksVer=1.30

cert_name=example-meow-${ranNum}

az group create -n ${rG} -l ${region} -o none

az aks create -n ${aks} -g ${rG} --kubernetes-version ${aksVer} --node-os-upgrade-channel None \
--node-vm-size Standard_A4_v2 --node-count 1 --enable-app-routing --no-ssh-key
infra_rG=$(az aks show -n ${aks} -g ${rG} --query nodeResourceGroup -o tsv)

# Section: set-policy mode
az keyvault create -n ${kv} -g ${rG} --enable-rbac-authorization false
az aks approuting update -n ${aks} -g ${rG} --enable-kv


# Grant permission
kvprovider_mi_client_id=$(az identity show --resource-group ${infra_rG} --name "azurekeyvaultsecretsprovider-${aks}" --query clientId -o tsv)
az keyvault set-policy -n ${kv} --certificate-permissions get --spn ${kvprovider_mi_client_id}

webapp_mi_client_id=$(az identity show --resource-group ${infra_rG} --name "webapprouting-${aks}" --query clientId -o tsv)
az keyvault set-policy -n ${kv} --certificate-permissions get --spn ${webapp_mi_client_id}

# Generate certificate
openssl req -new -x509 -nodes -subj "/CN=${cert_name}" -addext "subjectAltName=DNS:${cert_name}" -out ${cert_name}.crt -keyout ${cert_name}.key
openssl pkcs12 -export -in ${cert_name}.crt -inkey ${cert_name}.key -out ${cert_name}.pfx

# Import
az keyvault certificate import --vault-name ${kv} -n ${cert_name} -f ${cert_name}.pfx
certUrl=$(az keyvault certificate show --vault-name ${kv} -n ${cert_name} --query id -o tsv | sed -E 's/((.*)([\/]))([a-z0-9]+)/\2/')

# Get AKS credentials 
az aks get-credentials -n ${aks} -g ${rG}

# Apply default cert
cat <<EOF | kubectl apply -f -
apiVersion: approuting.kubernetes.azure.com/v1alpha1
kind: NginxIngressController
metadata:
  name: default
spec:
  ingressClassName: webapprouting.kubernetes.azure.com
  controllerNamePrefix: nginx
  loadBalancerAnnotations: 
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
  defaultSSLCertificate:
    keyVaultURI: ${certUrl}
EOF

JoeyC-Dev avatar Jul 24 '24 05:07 JoeyC-Dev