`secondaryCidrBlock` allocates nodes in Pods subnets depending on the order of `AWSManagedControlPlane.Spec.Network.Subnets` list

[esierra-stratio]

trafficstars

/kind bug

What steps did you take and what happened:

Set param secondaryCidrBlock and specify AWSManagedControlPlane.Spec.Network.Subnets in object AWSManagedControlPlane alternating subnets dedicated to nodes and pods.

When an instance is created it takes the first private subnet in the list for a specific AZ no matter if it is a secondary subnet (pods) or not (nodes).

What did you expect to happen:

An instance should take the first private subnet in the list for a specific AZ excluding those with the following tag: sigs.k8s.io/cluster-api-provider-aws/association=secondary

Anything else you would like to add:

IMO, a filter excluding the subnet with a specific tag (this case, sigs.k8s.io/cluster-api-provider-aws/association=secondary) is missing during ec2 creation here: https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/7eb4851dd9bdeacf039b6609176b81c068891d9a/pkg/cloud/services/ec2/instances.go#L357 Adding something like

subnets = subnets.ExcludeByTag(infrav1.NameAWSSubnetAssociation, infrav1.SecondarySubnetTagValue)

and

// FilterByTag returns a slice containing subnets that not contains specific tag.
func (s Subnets) ExcludeByTag(tagkey string, tagvalue string) (res Subnets) {
	for _, x := range s {
		if val, ok := x.Tags[tagkey]; !(ok && val == tagvalue) {
			res = append(res, x)
		}
	}
	return
}

in networks_types.go seems to work well.

Environment:

• Cluster-api-provider-aws version: v2.0.2
• Kubernetes version: (use kubectl version): v1.24.7
• OS (e.g. from /etc/os-release):

PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
UBUNTU_CODENAME=jammy