AKS Cluster must have at least one system pool unless it is being clusterctl-moved

[h0tbird]

/kind bug

What steps did you take and what happened:

1. Create a local kind cluster.
2. Set EXP_MACHINE_POOL=true and EXP_AKS=true.
3. clusterctl init with --infrastructure azure.
4. clusterctl generate cluster with --flavor aks.
5. Init the target cluster (same as 3).
6. clusterctl move from local kind to target AKS.
7. Notice the error below:

clusterctl -v 1 move -n little-sunshine --to-kubeconfig little-sunshine.kubeconfig --to-kubeconfig-context little-sunshine
Performing move...
Discovering Cluster API objects
Total objects Count=10
Moving Cluster API objects Clusters=1
Moving Cluster API objects ClusterClasses=0
Pausing the source cluster
Pausing the source cluster classes
Creating target namespaces, if missing
Creating Namespace="little-sunshine"
Creating objects in the target cluster
Creating AzureClusterIdentity="cluster-identity" Namespace="little-sunshine"
Creating Cluster="little-sunshine" Namespace="little-sunshine"
Creating AzureManagedCluster="little-sunshine" Namespace="little-sunshine"
Creating MachinePool="pool0" Namespace="little-sunshine"
Creating MachinePool="pool1" Namespace="little-sunshine"
Creating AzureManagedControlPlane="little-sunshine" Namespace="little-sunshine"
Creating AzureManagedMachinePool="pool1" Namespace="little-sunshine"
Creating Secret="little-sunshine-kubeconfig" Namespace="little-sunshine"
Creating AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting objects from the source cluster
Deleting AzureManagedMachinePool="pool1" Namespace="little-sunshine"
Deleting Secret="little-sunshine-kubeconfig" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Deleting AzureManagedMachinePool="pool0" Namespace="little-sunshine"
Error: action failed after 10 attempts: error deleting "infrastructure.cluster.x-k8s.io/v1beta1, Kind=AzureManagedMachinePool" little-sunshine/pool0: admission webhook "validation.azuremanagedmachinepools.infrastructure.cluster.x-k8s.io" denied the request: if the delete is triggered via owner MachinePool please refer to trouble shooting section in https://capz.sigs.k8s.io/topics/managedcluster.html: AKS Cluster must have at least one system pool

What did you expect to happen: I expect all AzureManagedMachinePool objects to be deleted from the local kind cluster and re-created in the target AKS cluster.

Anything else you would like to add: Having zero system pools should be fine during clusterctl move operations. Maybe the webhook can check wether the parent cluster object has the spec.paused set to true. Or maybe clusterctl move should annotate the objects it is moving and then webhooks would honour that. The error is raised here.

Environment:

• cluster-api-provider-azure version: v1.5.0
• Kubernetes version: (use kubectl version): v1.25.0

[CecileRobertMichon]

/area managedclusters

[CecileRobertMichon]

/help

[k8s-ci-robot]

@CecileRobertMichon: This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[CecileRobertMichon]

/milestone v1.6

[nawazkh]

/assign

[pkbhowmick]

Hi @nawazkh, Is there any update on it?

[nawazkh]

I haven't been able to make much progress on this, although I was able to reproduce it locally. Will unassign myself for now if anyone wants to pick it up, or I will get back on this later on.

[nawazkh]

/unassign

[pkbhowmick]

I am assigning it to me. I faced this issue locally and I think, I can give a try.

[pkbhowmick]

/assign

[CecileRobertMichon]

/milestone v1.8

[CecileRobertMichon]

@jackfrancis @dtzar should we prioritize this for milestone v1.8? it's a known bug for AKS clusters with clusterctl move and would be good to get it fixed before graduation

[jackfrancis]

@CecileRobertMichon makes sense. Is this a bug w/ our docs or our default templates, or an actual bug? We should definitely do a thorough audit of the AKS documentation as part of this dev cycle as well as we should expect more usage following official graduation.

[CecileRobertMichon]

it's an actual bug (bug in validating webhook to be precise)