azure-npm INCOMPLETELY BLOCKS ingress traffic from LoadBalancer

[atakadams]

What happened:

Connections to a loadbalancer service, which is routed to pods with networkpolicy restricting ingress traffic, sometimes fail with connection timeout. This happens only when the pods are deployed on multiple nodes. When the pods are deployed on the same node, or there is only one pod under the loadbalancer service, all the connections succeed.

What you expected to happen:
All the connections fail regardless on which nodes the pods are running on. At least, there should not be any unstable behaviors, which makes us difficult to troubleshoot networkpolicy issues, even if we should not block connections through loadbalancer by networkpolicy.

How to reproduce it:

1. Create an AKS cluster (az aks create) with the options below: (three nodes , Azure CNI, Azure Network Policy) -c 3 --network-plugin azure --network-policy azure

2. Create a loadbalancer and two pods. Pods are deployed on different nodes with antiAffinity. (You can use "test-template.yaml" in templates.zip )

3. Apply networkpolicy, which has no ingress rule, on the namespace where the pods are deployed into. (You can use "limit-inbound-policy.yaml" in the templates.zip above)

4. Repeat curl -m 2 http://LOADBALANCER-EXTERNAL-IP-ADDRESS from outside of the AKS cluster. The connection sometimes succeeds, sometimes fails. I recommend you to try at least 10+ times to see both behaviors.

Kubernetes Version:
v1.21.9

Kernel (e.g. uname -a):
5.4.0-1072-azure

Anything else we need to know?: I gathered iptables content.

content of the iptables, obtained with iptables-save command on kube-proxy pod on the first node. node0-iptables-save-withpolicy.txt

[vakalapa]

@atakadams What region is this issue happening? This is a known issue and is fixed in latest releases. We can release a fix early to your cluster/subscription. Please email details of cluster and/or subscription to [email protected]

Thanks.

[atakadams]

Hi @vakalapa , thank you for your quick response! I'll contact you later.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days

[github-actions[bot]]

Issue closed due to inactivity.