gRPC-multiprocessing
gRPC-multiprocessing copied to clipboard
Request distribution
Thanks for this example - it was very helpful.
I have a very similar use case with CPU intensive requests. Multiprocessing works well with one limitation: I see requests frequently being accepted by a process that's already busy and having to wait - while other processes are idle.
Did you encounter this? If so, did you find a work-around?
I did encounter this issue sometimes too. Unfortunately, I couldn't understand why my requests were routed towards busy process at what seems to be random intervals. Most of the times the "load balancing" worked, but I have no insights on how to adjust for those cases where the request is routed to a busy process.