Ingress rules reconciliation is not comparing equivalent sets

[r4f4]

trafficstars

/kind bug

What steps did you take and what happened:

When reconciling Security Group ingress rules, CAPA is comparing sets that are not equivalent. That happens because in this commit the ingress rules from the SDK are duplicated to not contain both a sourceSecurityGroupID and cidr blocks. However, when deriving the desired rules set the rules are generated with sourceSGs and cidrBlocks containing multiple items.

This results in sets that are always different and rules will be revoked and authorized in every reconcile loop.

What did you expect to happen:

Equivalent sets are compared.

Anything else you would like to add:

Here is an example in Openshift where the rules are continuously revoked/authorized: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_installer/8545/pull-ci-openshift-installer-master-e2e-aws-ovn-fips/1801221043054972928/artifacts/e2e-aws-ovn-fips/ipi-install-install/artifacts/.openshift_install-1718286343.log

time="2024-06-13T12:47:33Z" level=debug msg="I0613 12:47:33.242557     325 recorder.go:104] \"Revoked security group ingress rules [protocol=tcp/range=[6443-6443]/description=Kubernetes API protocol=tcp/range=[6443-6443]/description=Kubernetes API protocol=tcp/range=[6443-6443]/description=Kubernetes API protocol=udp/range=[6081-6081]/description=Port 6081 (UDP) for geneve protocol=udp/range=[6081-6081]/description=Port 6081 (UDP) for geneve protocol=udp/range=[9000-9999]/description=Port 9000-9999 for node ports (UDP) protocol=udp/range=[9000-9999]/description=Port 9000-9999 for node ports (UDP) protocol=tcp/range=[10257-10257]/description=controller-manager protocol=tcp/range=[10257-10257]/description=controller-manager protocol=tcp/range=[22623-22623]/description=MCS traffic from cluster network protocol=tcp/range=[22623-22623]/description=MCS traffic from cluster network protocol=udp/range=[30000-32767]/description=Service node ports (UDP) protocol=udp/range=[30000-32767]/description=Service node ports (UDP) protocol=udp/range=[4789-4789]/description=Port 4789 (UDP) for VXLAN protocol=udp/range=[4789-4789]/description=Port 4789 (UDP) for VXLAN protocol=icmp/range=[-1--1]/description=ICMP protocol=icmp/range=[-1--1]/description=ICMP protocol=tcp/range=[10259-10259]/description=kube-scheduler protocol=tcp/range=[10259-10259]/description=kube-scheduler protocol=tcp/range=[30000-32767]/description=Service node ports (TCP) protocol=tcp/range=[30000-32767]/description=Service node ports (TCP) protocol=tcp/range=[22-22]/description=Port 22 (TCP) protocol=tcp/range=[22-22]/description=Port 22 (TCP) protocol=tcp/range=[9000-9999]/description=Port 9000-9999 for node ports (TCP) protocol=tcp/range=[9000-9999]/description=Port 9000-9999 for node ports (TCP) protocol=50/range=[0-0]/description=ESP protocol=50/range=[0-0]/description=ESP protocol=tcp/range=[6441-6442]/description=Port 6441-6442 (TCP) for ovndb protocol=tcp/range=[6441-6442]/description=Port 6441-6442 (TCP) for ovndb protocol=udp/range=[4500-4500]/description=Port 4500 (UDP) for IKE NAT protocol=udp/range=[4500-4500]/description=Port 4500 (UDP) for IKE NAT protocol=udp/range=[500-500]/description=Port 500 (UDP) for IKE protocol=udp/range=[500-500]/description=Port 500 (UDP) for IKE] for SecurityGroup \\\"sg-0156151301759f178\\\"\" logger=\"events\" type=\"Normal\" object={\"kind\":\"AWSCluster\",\"namespace\":\"openshift-cluster-api-guests\",\"name\":\"ci-op-ivvhn8jx-281c5-qh925\",\"uid\":\"4979c842-0b95-43af-a304-a0d0f12fc6e7\",\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta2\",\"resourceVersion\":\"378\"} reason=\"SuccessfulRevokeSecurityGroupIngressRules\""
time="2024-06-13T12:47:33Z" level=debug msg="I0613 12:47:33.572338     325 securitygroups.go:193] \"Authorized ingress rules in security group\" controller=\"awscluster\" controllerGroup=\"infrastructure.cluster.x-k8s.io\" controllerKind=\"AWSCluster\" AWSCluster=\"openshift-cluster-api-guests/ci-op-ivvhn8jx-281c5-qh925\" namespace=\"openshift-cluster-api-guests\" name=\"ci-op-ivvhn8jx-281c5-qh925\" reconcileID=\"9c713630-1132-44e5-bf46-6005b9c49e31\" cluster=\"openshift-cluster-api-guests/ci-op-ivvhn8jx-281c5-qh925\" authorized-ingress-rules=[{\"description\":\"ICMP\",\"protocol\":\"icmp\",\"fromPort\":-1,\"toPort\":-1,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 22 (TCP)\",\"protocol\":\"tcp\",\"fromPort\":22,\"toPort\":22,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 4789 (UDP) for VXLAN\",\"protocol\":\"udp\",\"fromPort\":4789,\"toPort\":4789,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 6081 (UDP) for geneve\",\"protocol\":\"udp\",\"fromPort\":6081,\"toPort\":6081,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 500 (UDP) for IKE\",\"protocol\":\"udp\",\"fromPort\":500,\"toPort\":500,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 4500 (UDP) for IKE NAT\",\"protocol\":\"udp\",\"fromPort\":4500,\"toPort\":4500,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"ESP\",\"protocol\":\"50\",\"fromPort\":-1,\"toPort\":-1,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 6441-6442 (TCP) for ovndb\",\"protocol\":\"tcp\",\"fromPort\":6441,\"toPort\":6442,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 9000-9999 for node ports (TCP)\",\"protocol\":\"tcp\",\"fromPort\":9000,\"toPort\":9999,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Port 9000-9999 for node ports (UDP)\",\"protocol\":\"udp\",\"fromPort\":9000,\"toPort\":9999,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Service node ports (TCP)\",\"protocol\":\"tcp\",\"fromPort\":30000,\"toPort\":32767,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Service node ports (UDP)\",\"protocol\":\"udp\",\"fromPort\":30000,\"toPort\":32767,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"Kubernetes API\",\"protocol\":\"tcp\",\"fromPort\":6443,\"toPort\":6443,\"sourceSecurityGroupIds\":[\"sg-01220615d4fe14896\",\"sg-0156151301759f178\",\"sg-030767034bf574232\"]},{\"description\":\"MCS traffic from cluster network\",\"protocol\":\"tcp\",\"fromPort\":22623,\"toPort\":22623,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"],\"sourceSecurityGroupRoles\":[\"node\",\"controlplane\"]},{\"description\":\"controller-manager\",\"protocol\":\"tcp\",\"fromPort\":10257,\"toPort\":10257,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"],\"sourceSecurityGroupRoles\":[\"controlplane\",\"node\"]},{\"description\":\"kube-scheduler\",\"protocol\":\"tcp\",\"fromPort\":10259,\"toPort\":10259,\"sourceSecurityGroupIds\":[\"sg-0156151301759f178\",\"sg-030767034bf574232\"],\"sourceSecurityGroupRoles\":[\"controlplane\",\"node\"]}] security-group-id=\"sg-0156151301759f178\""

Notice here the difference in sourceSecurityGroupIds as well as the presence of sourceSecurityGroupRoles in the want group.

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):