apache
Feature request: Add basic client side load balancing for gRPC
There is no client side load balancing for gRPC module. All the requests land on one instance rather than distributing the load between multiple instances. Is it possible to add basic load balancing on client side e.g. round_robin or other pattern so request load can be distributed among instances?
https://pekko.apache.org/docs/pekko-grpc/current/client/details.html#load-balancing
Does this cover your use case? The docs seem to indicate that we rely on grpc-java, a separate lib, for a lot of this support. If the existing support does not suit you, can you indicate what extra you are looking for?
As mentioned in the link you shared, "When multiple endpoints are discovered for a gRPC client, currently one is selected and used for all outgoing calls". Ideally we should be able to send requests to both endpoints in a round_robin fashion.
Even if round_robin is provided as load balancing policy, only one host is receiving requests.
I see in the docs, "Client-side load balancing is desirable when you are using the default static or the grpc-dns discovery mechanism.". But the meaning of default static method is not clear to me. Can we add some example for this use case?
Also, I searched on internet for client side load balancing for akka-grpc with akka discovery and found old merge in akka grpc:(https://github.com/akka/akka-grpc/issues/809)
That akka-grpc PR is in Pekko gRPC. The fork was made after the merge was made.
indeed, but there is no clear doc/explaination on how to achieve client side load balancing, even with the default static mechanism as I mentioned in above comment.
I will repeat pekko-grpc is built on top of grpc-java and https://pekko.apache.org/docs/pekko-grpc/current/client/details.html#load-balancing highlights how to set the load-balancing-policy config. You can go and read the grpc-java docs to find out what the settings are (we already list round_robin as being supported). Apache Pekko is a free lib with no warranties or support. You can help by coming back with what you learn and add to the pekko-grpc docs to help future users.
As mentioned in the link you shared, "When multiple endpoints are discovered for a gRPC client, currently one is selected and used for all outgoing calls".
We should probably add a 'by default' there.
Ideally we should be able to send requests to both endpoints in a
round_robinfashion. Even ifround_robinis provided as load balancing policy, only one host is receiving requests.
As that documentation mentions, while perhaps counter-intuitive, there are situations where that's a better strategy - but I agree it would be good if client-side load balancing were possible.
I see in the docs, "Client-side load balancing is desirable when you are using the default static or the grpc-dns discovery mechanism.".
I suspect we should replace 'desirable' with 'possible' here.
But the meaning of default static method is not clear to me.
'static' is the discovery method that is used when you don't configure a different service discovery mechanism (https://pekko.apache.org/docs/pekko-grpc/current/client/configuration.html#using-pekko-discovery-for-endpoint-discovery).
Can we add some example for this use case?
I agree that would be helpful.
Even if
round_robinis provided as load balancing policy, only one host is receiving requests.
Indeed that seems to contradict the documentation. Could you boil down your application to a minimal test project to show the behavior?
