lettuce icon indicating copy to clipboard operation
lettuce copied to clipboard

Cost too much time between write() done and Closing Redis Connection

Open luxinSeu opened this issue 5 months ago • 6 comments

Bug Report

I do not find someone else has already reported your issue

Current Behavior

Get and set redis data slow(cost about 100ms), when faceing 5000 times query per second. When I read the debug log. I find lettuce cost too much time between write() done and Closing Redis Connection.

2025-08-01T17:15:14.465+08:00 DEBUG 20417 --- [tr069] [pool-6-thread-49] i.l.c.c.PooledClusterConnectionProvider  : getConnection(READ, 12904)
2025-08-01T17:15:14.465+08:00 DEBUG 20417 --- [tr069] [pool-6-thread-49] i.lettuce.core.protocol.DefaultEndpoint  : [channel=0x86685ab9, /178.28.201.148:56892 -> 178.28.201.227/178.28.201.227:6384, epid=0x3d] write() writeAndFlush command ClusterCommand [command=AsyncCommand [type=GET, output=ValueOutput [output=null, error='null'], commandType=io.lettuce.core.protocol.Command], redirections=0, maxRedirections=5]
**2025-08-01T17:15:14.465+08:00** DEBUG 20417 --- [tr069] [pool-6-thread-49] i.lettuce.core.protocol.DefaultEndpoint  : [channel=0x86685ab9, /178.28.201.148:56892 -> 178.28.201.227/178.28.201.227:6384, epid=0x3d] write() done
**2025-08-01T17:15:14.500+08:00** DEBUG 20417 --- [tr069] [pool-6-thread-49] o.s.d.redis.core.RedisConnectionUtils    : Closing Redis Connection
2025-08-01T17:15:14.376+08:00 DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] i.l.c.c.PooledClusterConnectionProvider  : getConnection(READ, 4650)
2025-08-01T17:15:14.376+08:00 DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] i.lettuce.core.protocol.DefaultEndpoint  : [channel=0x145b4a06, /178.28.201.148:54994 -> 178.28.201.148/178.28.201.148:6385, epid=0x3e] write() writeAndFlush command ClusterCommand [command=AsyncCommand [type=GET, output=ValueOutput [output=null, error='null'], commandType=io.lettuce.core.protocol.Command], redirections=0, maxRedirections=5]
**2025-08-01T17:15:14.376+08:00** DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] i.lettuce.core.protocol.DefaultEndpoint  : [channel=0x145b4a06, /178.28.201.148:54994 -> 178.28.201.148/178.28.201.148:6385, epid=0x3e] write() done
**2025-08-01T17:15:14.488+08:00** DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] o.s.d.redis.core.RedisConnectionUtils    : Closing Redis Connection

Input Code

String json = stringRedisTemplate.opsForValue().get(key);
application.yml:
  data:
    redis:
      host: 10.xx.xx.xxx
      port: 6379
      password: ****
      timeout: 5000ms
      lettuce:
        cluster:
          refresh:
            adaptive: true
            period: 30000
        pool:
          enabled: true
          max-active: 500
          max-idle: 500
          min-idle: 50
          max-wait: 5000ms

Expected behavior/code

Environment

  • Lettuce version(s): 6.4.2.RELEASE
  • Redis version: 6.2.7
  • Springboot :3.4.4

Possible Solution

NA

Additional context

NA

luxinSeu avatar Aug 04 '25 03:08 luxinSeu

Besides, I find that lettuce only use three thread on the log, I do not know why. here is one of the thread log, could you help figure out where is wrong ,thank you !

	行 171921: 2025-08-04T11:53:37.487+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.lettuce.core.protocol.CommandHandler   : [channel=0x8f9cfef8, /178.28.201.148:42874 -> 178.28.201.148/178.28.201.148:6388, epid=0x3e, chid=0x3d] Received: 8362 bytes, 108 commands in the stack
	行 171922: 2025-08-04T11:53:37.487+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.lettuce.core.protocol.CommandHandler   : [channel=0x8f9cfef8, /178.28.201.148:42874 -> 178.28.201.148/178.28.201.148:6388, epid=0x3e, chid=0x3d] Stack contains: 108 commands
	行 171923: 2025-08-04T11:53:37.487+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.l.core.protocol.RedisStateMachine      : Decode done, empty stack: true
	行 171924: 2025-08-04T11:53:37.487+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.lettuce.core.protocol.CommandHandler   : [channel=0x8f9cfef8, /178.28.201.148:42874 -> 178.28.201.148/178.28.201.148:6388, epid=0x3e, chid=0x3d] Completing command LatencyMeteredCommand [type=EXISTS, output=IntegerOutput [output=1, error='null'], commandType=io.lettuce.core.cluster.ClusterCommand]
	行 172635: 2025-08-04T11:53:37.487+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.lettuce.core.protocol.CommandHandler   : [channel=0x8f9cfef8, /178.28.201.148:42874 -> 178.28.201.148/178.28.201.148:6388, epid=0x3e, chid=0x3d] Stack contains: 107 commands
	行 172637: 2025-08-04T11:53:37.490+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.l.core.protocol.RedisStateMachine      : Decode done, empty stack: true
	行 172638: 2025-08-04T11:53:37.490+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.lettuce.core.protocol.CommandHandler   : [channel=0x8f9cfef8, /178.28.201.148:42874 -> 178.28.201.148/178.28.201.148:6388, epid=0x3e, chid=0x3d] Completing command LatencyMeteredCommand [type=GET, output=ValueOutput [output=[B@74483046, error='null'], commandType=io.lettuce.core.cluster.ClusterCommand]
	行 172639: 2025-08-04T11:53:37.490+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.lettuce.core.protocol.CommandHandler   : [channel=0x8f9cfef8, /178.28.201.148:42874 -> 178.28.201.148/178.28.201.148:6388, epid=0x3e, chid=0x3d] Stack contains: 106 commands
	行 172640: 2025-08-04T11:53:37.490+08:00 DEBUG 2495090 --- [tr069] [lettuce-epollEventLoop-4-5] i.l.core.protocol.RedisStateMachine      : Decode done, empty stack: true

luxinSeu avatar Aug 04 '25 07:08 luxinSeu

Hey @luxinSeu ,

we would need some more details to help out with this issue. Could you send us the whole code you are running, including creating / closing the connection to Redis? Are you setting up something else on the driver, besides the things in the configuration?

My biggest concern is - why is the connection being closed?

tishun avatar Aug 04 '25 10:08 tishun

Second question - why are you using a pool at all? Do you use blocking commands or transactions? See more info here.

tishun avatar Aug 04 '25 10:08 tishun

Thanks for your answer:

  1. I recently upgraded my springboot version, from 2.7.6 to 3.4.4. Before I upgraded ,the lettuce and redis works very fast.
  2. All configration is in application.yml, I have already listed.
  3. Since I do not change configration by code, so there is no code for creating / closing the connection to Redis
  4. why I use connecting pool is for improve performance(and it really works!), My program need to access to Redis more than 5000 tims per second for each server.
  5. I do not know why the connection being closed, I do not close connection by code. As I mentioned earlier, when I use this command, the following four lines logs are displayed. But the connection being closed every time I set or get from redis String json = stringRedisTemplate.opsForValue().get(key);

2025-08-01T17:15:14.376+08:00 DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] i.l.c.c.PooledClusterConnectionProvider : getConnection(READ, 4650) 2025-08-01T17:15:14.376+08:00 DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] i.lettuce.core.protocol.DefaultEndpoint : [channel=0x145b4a06, /178.28.201.148:54994 -> 178.28.201.148/178.28.201.148:6385, epid=0x3e] write() writeAndFlush command ClusterCommand [command=AsyncCommand [type=GET, output=ValueOutput [output=null, error='null'], commandType=io.lettuce.core.protocol.Command], redirections=0, maxRedirections=5] **2025-08-01T17:15:14.376+08:00** DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] i.lettuce.core.protocol.DefaultEndpoint : [channel=0x145b4a06, /178.28.201.148:54994 -> 178.28.201.148/178.28.201.148:6385, epid=0x3e] write() done **2025-08-01T17:15:14.488+08:00** DEBUG 20417 --- [tr069] [https-jsse-nio-9090-exec-468] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection 7. All Code is as follows:

package com.xxx.xxx.utils;
@Component
public class BasicCommonCacheUtils {
   private final static TenantLog log = TenantLog.getLogger(BasicCommonCacheUtils.class.getSimpleName());
   @Resource
   private StringRedisTemplate stringRedis;

   @Resource(name = "taskExecutor")
   private ThreadPoolTaskExecutor threadPoolTaskExe;

   private static StringRedisTemplate stringRedisTemplate;

   private static ThreadPoolTaskExecutor threadPoolTaskExecutor;

   @PostConstruct
   public void init() {
       stringRedisTemplate = stringRedis;
       threadPoolTaskExecutor = threadPoolTaskExe;
   }

   public static final String[] THREAD_STATUS = {"0", "1"};

   public static void set(final String key, final Object value) {
       try {
           stringRedisTemplate.opsForValue().set(key, CommonStbUtil.toJSONString(value));
       } catch (Exception e) {
           log.error("set key no expire failed =" + key, e);
       }
   }

   public static void set(String key, Object value, int time, TimeUnit unit) {
       try {
           stringRedisTemplate.opsForValue().set(key, CommonStbUtil.toJSONString(value), time, unit);
       } catch (Exception e) {
           log.error("set key failed =" + key, e);
       }
   }

   public static <T> T get(final String key, Class<T> type) {
       T result = null;
       try {
           String json = stringRedisTemplate.opsForValue().get(key);
           result = CommonStbUtil.parseObject(json, type);
       } catch (Exception e) {
           log.error("get T Exception", e);
       }
       return result;
   }

   public static String get(final String key) {
       String json = null;
       try {
           json = stringRedisTemplate.opsForValue().get(key);
       } catch (Exception e) {
           log.error("get Exception", e);
       }
       return json;
   }
}

luxinSeu avatar Aug 05 '25 11:08 luxinSeu

Besides, I used the dependency

<dependency>   
	<groupId>org.apache.commons</groupId>
	<artifactId>commons-pool2</artifactId>
	<version>2.12.0</version>
</dependency>   

luxinSeu avatar Aug 05 '25 11:08 luxinSeu

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 2 weeks this issue will be closed.

github-actions[bot] avatar Sep 05 '25 00:09 github-actions[bot]