Rotate KMS key cause cluster stuck in updating status

[RichardChen820]

Describe the bug We are planning to use KMS v2 to encrypt secret data at rest on our production AKS cluster, When I doing the testing, I just found that rotating KMS key causes the cluster stuck in updating status.

Steps To Reproduce

1. Enable KMS v2 with key version#1, and update the existing secret to be replaced as encrypted data
2. Create a key version#2 and update keyID of KMS plugin to use version#2, but do NOT re-encrypt the secret
3. Create a key version#3 and update keyID of KMS plugin to use version#3
4. See cluster stuck in updating status

KMS Plugin for Key Vault version Default version on AKS v1.27.3

Kubernetes version AKS v1.27.3

Additional context

[RichardChen820]

Update command used in repro step 2 and 3

az aks update --name $mycluster--resource-group $myRG --enable-azure-keyvault-kms --azure-keyvault-kms-key-vault-network-access "Public" --azure-keyvault-kms-key-id $keyVaultKeyId

[enj]

This issue refers to an AKS limitation. Please track https://github.com/Azure/AKS/issues/4648 for updates.