dubbo icon indicating copy to clipboard operation
dubbo copied to clipboard

#10227 优化subscribe时初始化性能,经过优化后,该场景在开发环境下从5~20秒性能提升至300~1000ms,提升5~20倍,基本上在30…

Open tietang opened this issue 2 years ago • 2 comments

What is the purpose of the change

原来为使用redis scan命令来扫描 /dubbo/com.xxx.yyy.XxxService/*的key 按照https://dubbo.apache.org/zh/docsv2.7/user/references/registry/redis/对redis注册中心的介绍 只有/dubbo/com.xxx.yyy.XxxService/providers 和/dubbo/com.xxx.yyy.XxxService/consumers 2种可能性。 在在使用ClusterRedisClient时,scan命令(参考:org.apache.dubbo.remoting.redis.jedis.ClusterRedisClient.scan)是迭代循环所有cluster node 来扫描每一个node,并合并扫描结果完成的,scan命令本来就很慢,使用ClusterRedisClient时会随着节点越多越慢 我的场景中有6个节点,每次需要5 ~ 20秒的扫描时间,每次启动时很慢且卡在初始化reference阶段;或者泛化每接口第一次调用或者硬编码每接口第一次调用时,都会构建Reference,构建reference时花费了很多时间 如果项目中service 接口很多时,有可能在启动或者重启后会导致worker线程阻塞而请求无法正常调用,目前在测试环境已经遇到了该现象,高并发请求时有可能引起雪奔,特别是存在dubbo http网关时的场景。 经过优化后,该场景在开发环境下从5 ~ 20秒性能提升至300~1000ms,提升5 ~ 20倍,基本上在300 ~ 1000ms即可构建好reference。 测试环境多线程(50线程)预创建48个reference需要,同机房1985ms(优化前9839ms)提升5倍,跨机房(同地域)3276ms(优化前35303ms)提升10倍。 本地环境(远程连接测试环境redis)多线程(50线程)预创建48个reference需要18288ms(优化前436235ms,约7分钟多,提升23倍)。

tietang avatar Jun 29 '22 02:06 tietang

#10227

tietang avatar Jun 29 '22 02:06 tietang

Codecov Report

Merging #10234 (d3d1075) into master (c1a76a0) will decrease coverage by 0.06%. The diff coverage is 66.66%.

@@             Coverage Diff              @@
##             master   #10234      +/-   ##
============================================
- Coverage     60.95%   60.89%   -0.07%     
+ Complexity      448      446       -2     
============================================
  Files          1100     1100              
  Lines         44579    44531      -48     
  Branches       6493     6488       -5     
============================================
- Hits          27175    27119      -56     
- Misses        14437    14444       +7     
- Partials       2967     2968       +1     
Impacted Files Coverage Δ
...dubbo/remoting/redis/jedis/ClusterRedisClient.java 0.00% <0.00%> (ø)
...ubbo/remoting/redis/jedis/SentinelRedisClient.java 0.00% <0.00%> (ø)
...org/apache/dubbo/registry/redis/RedisRegistry.java 59.20% <83.33%> (+0.77%) :arrow_up:
...he/dubbo/remoting/redis/jedis/MonoRedisClient.java 74.41% <100.00%> (-7.64%) :arrow_down:
...bo/remoting/redis/support/AbstractRedisClient.java 21.05% <0.00%> (-26.32%) :arrow_down:
...in/java/org/apache/dubbo/common/utils/JVMUtil.java 81.13% <0.00%> (-11.33%) :arrow_down:
...e/dubbo/remoting/transport/netty/NettyChannel.java 52.27% <0.00%> (-7.96%) :arrow_down:
...ng/transport/dispatcher/all/AllChannelHandler.java 89.65% <0.00%> (-6.90%) :arrow_down:
...he/dubbo/remoting/transport/netty/NettyServer.java 70.17% <0.00%> (-3.51%) :arrow_down:
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c1a76a0...d3d1075. Read the comment docs.

codecov-commenter avatar Jul 11 '22 07:07 codecov-commenter

2.6.x has been announced as EOF. Please file another pr to 2.7.x or 3.1.x and fix the issues mentioned in the comments.

chickenlj avatar Sep 02 '22 05:09 chickenlj