lettuce icon indicating copy to clipboard operation
lettuce copied to clipboard

Request queue size is not cleared on reconnect

Open Hixon10 opened this issue 2 years ago • 1 comments

Bug Report

  1. There is a Java (spring) service, which uses Redis cluster with help lettuce.
  2. We have a setting spring.redis.lettuce.pool.maxActive == 100_000
  3. At some moment, the service lose network connectivity to one of redis cluster master nodes.
  4. The service has a bunch of background queries, which are executed periodically (e.g. we execute ping query each second).
  5. After some time, io.lettuce.core.protocol.DefaultEndpoint#QUEUE_SIZE is equal to 100_000 for some channels
  6. As result, the service cannot execute any queries, using this connection. We got an error, like Request queue size exceeded: 100000. Commands are not accepted until the queue size drops.
  7. After some time, the network connective issue is resolved.
  8. The problem is, that the application still has io.lettuce.core.protocol.DefaultEndpoint#QUEUE_SIZE = 100_000. If I manually set this field to 0, the application starts working.

Environment

  • Lettuce version(s): 6.1.2.RELEASE
  • Redis version: 6.2.5

Maybe, it is similar to https://github.com/lettuce-io/lettuce-core/issues/616

Hixon10 avatar Sep 04 '21 18:09 Hixon10

Lettuce auto-reconnects and replays commands upon reconnect and that should drain the queue.

mp911de avatar Sep 08 '21 08:09 mp911de