dubbo
dubbo copied to clipboard
k8s环境下,provider重新部署后,consumer一直重连老的provider ip
- [ ] I have searched the issues of this repository and believe that this is not a duplicate.
Environment
- spring boot: 2.4.10
- zookeeper: 3.6.4
- Dubbo version: 3.0.3
- Java version: 1.8
Steps to reproduce this issue
1、腾讯云k8s上重新部署provider,此时provider pod重建后,ip由192.168.1.207 变成 192.168.1.208 2、consumer的RecollectTimerTask一直尝试连接provider的192.168.1.207
错误信息如下
2021-12-11 22:17:46.137|ERROR|ubbo-client-idleCheck-thread-1|o.a.d.r.e.s.header.ReconnectTimerTask | [DUBBO] Fail to connect to HeaderExchangeClient [channel=org.apache.dubbo.remoting.transport.netty4.NettyClient [/192.168.1.77:39932 -> /192.168.1.207:20880]], dubbo version: 3.0.3, current host: 192.168.1.77 org.apache.dubbo.remoting.RemotingException: client(url: dubbo://192.168.1.207:20880/com.fs.service.product.TradeProductService?anyhost=true&application=product-service&check=false&codec=dubbo&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&group=test&heartbeat=60000&interface=com.fs.service.product.TradeProductService&metadata-type=remote&methods=approveById,queryByPage,removeById,create,updateById&release=3.0.3&revision=1.0-SNAPSHOT&service-name-mapping=true&service.filter=threadExecutorFilter&side=provider&timeout=6000&token=92f77e4c-37fe-450e-be3f-96e66eb9e71b&version=1.0.0.test) failed to connect to server /192.168.1.207:20880 client-side timeout 3000ms (elapsed: 3001ms) from netty client 192.168.1.77 using dubbo version 3.0.3 at org.apache.dubbo.remoting.transport.netty4.NettyClient.doConnect(NettyClient.java:175) at org.apache.dubbo.remoting.transport.AbstractClient.connect(AbstractClient.java:202) at org.apache.dubbo.remoting.transport.AbstractClient.reconnect(AbstractClient.java:258) at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeClient.reconnect(HeaderExchangeClient.java:166) at org.apache.dubbo.remoting.exchange.support.header.ReconnectTimerTask.doTask(ReconnectTimerTask.java:49) at org.apache.dubbo.remoting.exchange.support.header.AbstractTimerTask.run(AbstractTimerTask.java:87) at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:648) at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:727) at org.apache.dubbo.common.timer.HashedWheelTimer$Worker.run(HashedWheelTimer.java:449) at java.lang.Thread.run(Thread.java:748)
期望
请问如何解决consumer一直刷Fail to connect to HeaderExchangeClient的问题,在其他issue中也看到很多类似这样的bug但是都没有解决方案,有说3.0.3好使的,但是从3.0.2.1升级到3.0.3还是不好使。
k8s里面不应该连接svc的IP吗,为什么要连接pod的ip?
用的什么注册中心配置?需要确认下有没有收到最新的地址通知
我也遇到这个问题,该怎么解决
用的什么注册中心配置?需要确认下有没有收到最新的地址通知
我也遇到这个问题,用的nacos注册中心配置 服务提供方重新部署后会有短暂的连接之前的ip报错,然后才会切换过来 感觉是不是nacos1.x 不支持长链接 推送失败导致的
用的什么注册中心配置?需要确认下有没有收到最新的地址通知
我也遇到这个问题,用的nacos注册中心配置 服务提供方重新部署后会有短暂的连接之前的ip报错,然后才会切换过来 感觉是不是nacos1.x 不支持长链接 推送失败导致的 用的nacos 2.0.3,现象就是旧的pod会推送消息给consumer,但是consumer还是连就是pod而不会去连新的pod
dubbo 2.7.12 nacos:2.x 也会遇到这个问题,当provider部署后,consumer就报错:Host is unreachable,然后重启consumer就好了
- [ ] I have searched the issues of this repository and believe that this is not a duplicate.
Environment
- spring boot: 2.4.10
- zookeeper: 3.6.4
- Dubbo version: 3.0.3
- Java version: 1.8
Steps to reproduce this issue
1、腾讯云k8s上重新部署provider,此时provider pod重建后,ip由192.168.1.207 变成 192.168.1.208 2、consumer的RecollectTimerTask一直尝试连接provider的192.168.1.207
错误信息如下
2021-12-11 22:17:46.137|ERROR|ubbo-client-idleCheck-thread-1|o.a.d.r.e.s.header.ReconnectTimerTask | [DUBBO] Fail to connect to HeaderExchangeClient [channel=org.apache.dubbo.remoting.transport.netty4.NettyClient [/192.168.1.77:39932 -> /192.168.1.207:20880]], dubbo version: 3.0.3, current host: 192.168.1.77 org.apache.dubbo.remoting.RemotingException: client(url: dubbo://192.168.1.207:20880/com.fs.service.product.TradeProductService?anyhost=true&application=product-service&check=false&codec=dubbo&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&group=test&heartbeat=60000&interface=com.fs.service.product.TradeProductService&metadata-type=remote&methods=approveById,queryByPage,removeById,create,updateById&release=3.0.3&revision=1.0-SNAPSHOT&service-name-mapping=true&service.filter=threadExecutorFilter&side=provider&timeout=6000&token=92f77e4c-37fe-450e-be3f-96e66eb9e71b&version=1.0.0.test) failed to connect to server /192.168.1.207:20880 client-side timeout 3000ms (elapsed: 3001ms) from netty client 192.168.1.77 using dubbo version 3.0.3 at org.apache.dubbo.remoting.transport.netty4.NettyClient.doConnect(NettyClient.java:175) at org.apache.dubbo.remoting.transport.AbstractClient.connect(AbstractClient.java:202) at org.apache.dubbo.remoting.transport.AbstractClient.reconnect(AbstractClient.java:258) at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeClient.reconnect(HeaderExchangeClient.java:166) at org.apache.dubbo.remoting.exchange.support.header.ReconnectTimerTask.doTask(ReconnectTimerTask.java:49) at org.apache.dubbo.remoting.exchange.support.header.AbstractTimerTask.run(AbstractTimerTask.java:87) at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:648) at org.apache.dubbo.common.timer.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:727) at org.apache.dubbo.common.timer.HashedWheelTimer$Worker.run(HashedWheelTimer.java:449) at java.lang.Thread.run(Thread.java:748)
期望
请问如何解决consumer一直刷Fail to connect to HeaderExchangeClient的问题,在其他issue中也看到很多类似这样的bug但是都没有解决方案,有说3.0.3好使的,但是从3.0.2.1升级到3.0.3还是不好使。
请问这个问题解决了吗,我也遇到同样的问题
3.1.0 仍有同样的问题
May related with #10570
dubbo 2.7.12 nacos:2.x 也会遇到这个问题,当provider部署后,consumer就报错:Host is unreachable,然后重启consumer就好了
请问,大家之前的这个问题定位到原因、解决了吗? 感谢
May be fixed in https://github.com/apache/dubbo/pull/10938. Please feel free to reopen or create a new PR if you have any question.