nacos icon indicating copy to clipboard operation
nacos copied to clipboard
verified publisher icon alibaba

error: Cannot create stream 545225 greater than Last-Stream-ID 545223 from GOAWAY.

Open ranLee1 opened this issue 1 year ago • 0 comments

Describe the bug Automatic retry registration failed, restart the client program to register Occasionally, the gateway cannot access the subservice until the gateway times out.😢 Server reports an error:

Cannot create stream 545225 greater than Last-Stream-ID 545223 from GOAWAY.

Client reports an error:

Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=0,errorMessage=Client not connected,current status:UNHEALTHY
Send request fail, request=ServiceQueryRequest{headers={}, requestId='null'}, retryTimes=0,errorMessage=java.util.concurrent.TimeoutException: Waited 3000 milliseconds (plus 1751 milliseconds, 965480 nanoseconds delay) but future completed as timeout expired

Expected behavior none Actually behavior none How to Reproduce none

Desktop (please complete the following information):

  • OS: CentOS Linux release 7.9.2009 (Core)
  • Version nacos-server 2.0.3, nacos-client 2.0.3
  • SDK spring-cloud-alibaba-nacos-config:2.2.7.RELEASE,spring-cloud-alibaba-nacos-discovery:2.2.7.RELEASE, dubbo:2.7.13

Additional context nacos-service:

2024-06-28 10:06:20,454 WARN Stream Error

io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception$StreamException: Cannot create stream 545225 greater than Last-Stream-ID 545223 from GOAWAY.
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:147)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultEndpoint.checkNewStreamAllowed(DefaultHttp2Connection.java:874)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultEndpoint.createStream(DefaultHttp2Connection.java:748)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultEndpoint.createStream(DefaultHttp2Connection.java:668)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(DefaultHttp2ConnectionDecoder.java:330)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onHeadersRead(Http2InboundFrameLogger.java:65)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader$1.processFragment(DefaultHttp2FrameReader.java:457)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readHeadersFrame(DefaultHttp2FrameReader.java:464)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:254)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:160)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:174)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:378)
	at io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)
	at io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:794)
	at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:424)
	at io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326)
	at io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
	at io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:748)

nacos-client:

2024-06-28 10:06:31.360  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.0[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=0,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:06:32.214  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.0[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=1,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:06:32.214  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.1[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=0,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:06:33.212  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.1[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=1,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:06:33.212  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.0[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=2,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:06:53.154  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.1[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={}, requestId='null'}, retryTimes=0,errorMessage=java.util.concurrent.TimeoutException: Waited 3000 milliseconds (plus 1751 milliseconds, 965480 nanoseconds delay) but future completed as timeout expired
2024-06-28 10:06:34.043  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.1[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=2,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:06:34.043  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.0[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=0,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:07:42.720  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.1[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=0,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:08:50.275  [1;31mERROR[m [[mcom.alibaba.nacos.naming.failover[m] [36mcom.alibaba.nacos.client.naming.backups.FailoverReactor$SwitchRefresher[m: [NA] failed to read failover switch.
[31m java.lang.OutOfMemoryError: GC overhead limit exceeded
[m2024-06-28 10:08:16.558  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.1[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=0,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:08:41.950  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.0[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={app=unknown}, requestId='null'}, retryTimes=0,errorMessage=Client not connected,current status:UNHEALTHY
2024-06-28 10:08:19.358  [1;31mERROR[m [[mhttp-nio-6803-exec-33[m] [36morg.apache.juli.logging.DirectJDKLog[m: Failed to complete processing of a request
[31m java.lang.OutOfMemoryError: GC overhead limit exceeded
[m2024-06-28 10:07:26.297  [1;31mERROR[m [[mcom.alibaba.nacos.client.naming.updater.0[m] [36mcom.alibaba.nacos.common.utils.LoggerUtils[m: Send request fail, request=ServiceQueryRequest{headers={}, requestId='null'}, retryTimes=0,errorMessage=java.util.concurrent.TimeoutException: Waited 3000 milliseconds (plus 478 milliseconds, 878195 nanoseconds delay) but future completed as timeout expired

ranLee1 avatar Jun 28 '24 06:06 ranLee1