jedis icon indicating copy to clipboard operation
jedis copied to clipboard

JedisSlotBasedConnectionHandler should wait until slotCache is renewed

Open kutzi opened this issue 6 years ago • 2 comments

Expected behavior

JedisSlotBasedConnectionHandler.getConnectionFromSlot() should always return a connection

Actual behavior

Sometimes - probaby during renewSlotCache() - no connection will be returned - even if the cluster is healthy. This seems to happen regularly shortly after our application startup - i.e. probably because the slot cache wasn't initialised, yet?

redis.clients.jedis.exceptions.JedisNoReachableClusterNodeException: No reachable node in cluster
	at redis.clients.jedis.JedisSlotBasedConnectionHandler.getConnection(JedisSlotBasedConnectionHandler.java:57) ~[jedis-2.9.0.jar:na]
	at redis.clients.jedis.JedisSlotBasedConnectionHandler.getConnectionFromSlot(JedisSlotBasedConnectionHandler.java:74) ~[jedis-2.9.0.jar:na]
	at redis.clients.jedis.JedisClusterCommand.runWithRetries(JedisClusterCommand.java:116) ~[jedis-2.9.0.jar:na]
	at redis.clients.jedis.JedisClusterCommand.runBinary(JedisClusterCommand.java:60) ~[jedis-2.9.0.jar:na]
	at redis.clients.jedis.BinaryJedisCluster.get(BinaryJedisCluster.java:98) ~[jedis-2.9.0.jar:na]

Steps to reproduce:

Couldn't reproduce reliably

Redis / Jedis Configuration

Redis 4.0.2 with a cluster of 3 masters and 3 slaves

Jedis version:

Jedis 2.9.0

Description:

If JedisSlotBasedConnectionHandler.getConnectionFromSlot() couldn't find a connection from the slot cache, it will trigger renewClusterSlots(), but will not wait for the end of the renewal, if the current thread didn't trigger the renewal itself. So getting a connection from the cache, again, and even falling back to the getConnection() method will fail, as the slots cache is cleared as one of the 1st steps in the renewal process.

Possible fixes

  • wait for the renewal to finish
  • don't clear the slots cache until it's completely renewed, so threads accessing it concurrently can at least get 'some' connection.

kutzi avatar Aug 20 '18 13:08 kutzi

See https://github.com/xetorthio/jedis/pull/1857 where I've implemented the 2nd proposal

kutzi avatar Aug 20 '18 14:08 kutzi

This issue is marked stale. It will be closed in 30 days if it is not updated.

github-actions[bot] avatar Feb 15 '24 00:02 github-actions[bot]