grpc-ecosystem
Getting too many pings enhance calm on spring GRPC client
We are using the spring boot starter to create server and client side implementation for GRPC service. After we start the GRPC server service and use the client the query service after few seconds in client logs we see message TOO_MANY_PINGS enhance calm. As per the documentation. The property keepAliveWithoutCalls should not be set in both client and server. In spring boot GRPC, the server set the property to false and client still sends ping to the server. What is the recommended values for the properties like keep alive time client/server and how can we change the properties.
The properties can be set like any spring-boot property using the configuration:
- https://yidongnan.github.io/grpc-spring-boot-starter/en/server/configuration.html#configuration-via-properties
- https://yidongnan.github.io/grpc-spring-boot-starter/en/client/configuration.html#configuration-via-properties
Our default values were selected according the defaults provided by grpc (which might have changed since then though).
There are some limits on how short you can/should set the keep-alive client side before the server complains.
The server doesn't send keep-alives AFAIK. This library uses false as the default for the client side keepAliveWithoutCalls.
Have you configured the keep alive at all on the server and the client?
@ST-DDT Right now the jar files for GRPC spring boot has default values for keep alive timeouts??? I have not changed the values, is it possible you can share the default values for timeouts.
The defaults are defined here: https://github.com/yidongnan/grpc-spring-boot-starter/blob/master/grpc-client-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/client/config/GrpcChannelProperties.java#L55 https://github.com/yidongnan/grpc-spring-boot-starter/blob/master/grpc-server-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/server/config/GrpcServerProperties.java#L57
@ST-DDT thanks. What I understand the server can abruptly close client connection based on defaults. Is there way we can set the following values at server and channel level Server side ('grpc.keepalive_time_ms', 10000), # send keepalive ping every 10 second, default is 2 hours ('grpc.keepalive_timeout_ms', 5000), # keepalive ping time out after 5 seconds, default is 20 seoncds ('grpc.keepalive_permit_without_calls', True), # allow keepalive pings when there's no gRPC calls ('grpc.http2.max_pings_without_data', 0), # allow unlimited amount of keepalive pings without data ('grpc.http2.min_time_between_pings_ms', 10000), # allow grpc pings from client every 10 seconds ('grpc.http2.min_ping_interval_without_data_ms', 5000), # allow grpc pings from client without data every 5 seconds
Client side 'grpc.keepalive_time_ms': 10000, 'grpc.keepalive_timeout_ms': 5000, 'grpc.keepalive_permit_without_calls': true, 'grpc.http2.max_pings_without_data': 0, 'grpc.http2.min_time_between_pings_ms':10000, 'grpc.http2.min_ping_interval_without_data_ms': 5000
Please check yourself which of these properties can be set via grpc's API and which of them we currently support:
- https://github.com/yidongnan/grpc-spring-boot-starter/blob/master/grpc-server-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/server/serverfactory/ShadedNettyGrpcServerFactory.java#L87
- https://github.com/yidongnan/grpc-spring-boot-starter/blob/master/grpc-client-spring-boot-autoconfigure/src/main/java/net/devh/boot/grpc/client/channelfactory/AbstractChannelFactory.java#L167
Please note, that you can customize the server/channels using custom beans:
- https://yidongnan.github.io/grpc-spring-boot-starter/en/server/configuration.html#grpcserverconfigurer
- https://yidongnan.github.io/grpc-spring-boot-starter/en/client/configuration.html#grpcchannelconfigurer
