Load Balancing among service endpoints

[sujitknandan]

Knative Serving version v1.7

We are finding Request are not getting distributed to all available pods behind the service. This is happening both when activator is present/absent in the data path.

I need to know how the different actors is working in both mode when Knative Service Setting (activator on only during start from zero)

metadata: annotations: autoscaling.knative.dev/target-burst-capacity: "0" spec: containerConcurrency: 1

Knative Service Setting (activator always present)

metadata: annotations: autoscaling.knative.dev/target-burst-capacity: "-1" spec: containerConcurrency: 1

When activator is inactive does load balancing completely depends upon kube-proxy, if that's is the case i tried with both iptables and ipvs mode in EKS v1.22 for kube-proxy. But result is more or less the same, it is not able to distribute properly.

When activator is active sometimes distribution works as expected sometime not. Does activator ensures that two request does not go to the same endpoint when containerConcurrency is set to 1, we are finding second request getting pushed to a backend which is presently serving a request