Does server handle requests from the same connction sequentially in one thread?

[abel-von]

Hi, I am rewriting my grpc server code from golang to rust and I'm planning to use grpc-rs, but when i compare the concurrency benchmark, the rust version is almost 10 times slower than golang version.

the client is using one connection to server, to do grpc call in many goroutines, so client is sending requests concurrently, but when I add some log in the server code, I find out that the server is handling the requests one by one, the the thread id are the same one.

change the parameter of Environment.new() do not change this.

So I guess is it because that grpc server is handling the request from the same connection sequentially in one thread? the concurrency only happens on the multiple connection?

[BusyJay]

Yes. See https://github.com/tikv/grpc-rs/issues/297#issuecomment-474261201.

[abel-von]

@BusyJay thank you very much, is making a spawn call in the method the only workaround? If it is that way, I have to write spawn call in every methods of grpc server implemetion, that makes codes too ugly.

[BusyJay]

Generally you need a thread pool and spawn time consuming task to them.

[Ten0]

I'm also hitting issues due to this (specifically, the server does not hear answers to its keepalive pings while the function call is busy, consequently stops sending new ones and we get cut off by our load balancer). I would use futures_cpupool, but I have several issues with this:

• spawn_fn requires that F is Send + 'static, and RpcContext has neither of these two.
• It does not provide an equivalent for turning iterators into streams of answers

What would you recommend to circumvent these issues ?

[BusyJay]

RpcContext is supposed to be used within gRPC context. futures_cpupool or tokio_threadpool both can be used for spawning futures.

Usually heavy computation not related to network should be offload to other thread pool, and only leave encoding/decoding to the gRPC. You can connect them with channels. TiKV project has a lot of such usage pattern you can refer to, especially the src/server/service/kv.rs file.