Jedis not releasing connections and keeping them in active state for ever

[venukbh]

Expected behavior

The connections should released from active state and returned to the pool or either go to broken connection pool as time out and then cleaned up

Actual behavior

The connections are staying in active pool for ever. We could see them using the jmx (jconsole) monitor and going to org.apache.commons.pool2 tab. It clearly shows number of active as 7 number of idle as 1 But then going to the list of objects, (7+1), we can see those active connections are not really active and they were last returned back to pool way long time it self.

End impact: As the active pool connections keeps on increasing, at some point when they reach the max pool and no more connections, the threads are spiking and the application is going down

All attributes of pool:

Connection 1 which is not used but remained in active since 5 days

Connection 2 which is not used but remained in active since 5 days

Connection 3 which is in use (this is the only active connection being used)

Connection 4 which is not used but remained in active since 5 days

Connection 5 which is not used but remained in active since 5 days

Connection 6 which is not used but remained in active since 5 days

Connection 7 which is not used but remained in active since 5 days

Connection 8 which is not used but remained in active since 5 days

Steps to reproduce:

No steps since there were no exceptions from any api - jedis or apache commons pool 2, or the others.

Redis / Jedis Configuration

configuration 1 which did not work

JedisPoolConfig.setMaxTotal(50);
JedisPoolConfig.setMaxIdle(2);
JedisPoolConfig.setMinIdle(0);
JedisPoolConfig.setMaxWaitMillis(2000); // 2 seconds

JedisPool(JedisPoolConfig, redis-1234.cluster-redislab.com, 1234, true)
Those parameters are
1. JedisPoolConfig
2. host
3. port
4. SSL = true

configuration 2 which either did not work

JedisPoolConfig.setMaxTotal(50);
JedisPoolConfig.setMaxIdle(2);
JedisPoolConfig.setMinIdle(0);
JedisPoolConfig.setMaxWaitMillis(2000); // 2 seconds

JedisPool(JedisPoolConfig, redis-1234.cluster-redislab.com, 1234, 5, true)
Those parameters are
1. JedisPoolConfig
2. host
3. port
4. Socket time out = 5 seconds
5. SSL = true

Jedis version:

3.0.0-m1

Redis version:

Redis Labs Enterprise Cluster 5.4.2-20 Latest open source Redis version supported 5.0.4 Latest open source Memcached version supported 1.4.17

Java version:

1.8.0_181

[venukbh]

I've opened a PR to fix this issue #2054

This will allow us to configure the abandon connections so that leaked connections can be closed in a timely fashion

[maksymkovalenko]

I suspect you're using Jedis incorrectly. What's your code that leads eventually runs into the issue? Jedis pool is Closeable and you must close it explicitly just like with any other resource (InputStream, Reader, JDBC, etc) I.e. should be for Java 8

try (JedisPool jedisPool = ...) {
  try (Jedis jedis = jedisPool.getResource()) {
    doStuff(jedis);
  }
}

private void doStuff(Jedis jedis) {
  jedis.set("myKey", "myVal");
  jedis.get("myKey");
}

Or pre java 8:

JedisPool jedisPool = ...;
try {
  Jedis jedis = jedisPool.getResource();
  try {
    doStuff(jedis);
  } finally {
    jedis.close();
  }
} finally {
  jedisPool.close();
}

I believe this issue should be closed.

[venukbh]

@maksymkovalenko I am already using java 8, and try with resource. If you see the images (first image and 2nd image in the order) I added above, you can see the connections being returned and borrowed, which means the connections are being closed i.e. returned to the pool. The issue still exist and should not be closed

[maksymkovalenko]

I see. You're using 3.0.0-m1. Is this still an issue in the latest 3.1.0? Another issue is that Jedis is using commons-pool2 2.6.2 whereas latest is 2.7.0. I'd say both need to be upgraded before doing anything else.

[maksymkovalenko]

In any case @venukbh , you should post your code, just describing the issue is not enough for us to do any kind of troubleshooting.

[venukbh]

@maksymkovalenko Yes, the issue is still with the latest versions of jedis clients which also brings commons-pool2 - 2.7.0

[maksymkovalenko]

Need code, otherewise there's nothing to inspect, debug, fix, etc.

[venukbh]

@maksymkovalenko

Here is the code, hope you can see the pool configuration above

public byte[] read(String key) { byte[] data = null; try (Jedis conn = getJedisPool().getResource()) { serializedData = conn.get(id.getBytes()); } catch (JedisException e) { // logging the exception } return data; }

[maksymkovalenko]

From what I see in your code, you're closing result of .getResource(), but you're not closing result of .getJedisPool().

On Tue, Oct 15, 2019 at 3:25 PM Venu [email protected] wrote:

@maksymkovalenko https://github.com/maksymkovalenko

Here is the code, hope you can see the pool configuration above

public byte[] read(String key) { byte[] data = null; try (Jedis conn = getJedisPool().getResource()) { serializedData = conn.get(id.getBytes()); } catch (JedisException e) { // logging the exception } return data; }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xetorthio/jedis/issues/2050?email_source=notifications&email_token=ACXCXWPPAMQT6OOEMZFKXXLQOY7PNA5CNFSM4ISEDBH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBKNMGA#issuecomment-542430744, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACXCXWP5GMMKVYO43HRX5KLQOY7PNANCNFSM4ISEDBHQ .

-- Max Kovalenko MileZero http://www.milezero.com/ | 206.349.0871 <//206.349.0871> 615 2nd Ave, Ste 150, Seattle, WA 98104

[venukbh]

@maksymkovalenko We should never close pool. Closing pool will close all the connections in the pool. We should only close resource, that way it will return back the connection to the pool.

[maksymkovalenko]

sure

On Tue, Oct 29, 2019 at 10:14 AM Venu [email protected] wrote:

@maksymkovalenko https://github.com/maksymkovalenko We should never close pool. Closing pool will close all the connections in the pool. We should only close resource, that way it will return back the connection to the pool.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/xetorthio/jedis/issues/2050?email_source=notifications&email_token=ACXCXWNJYUSPI3TEO22VUYDQRBVN3A5CNFSM4ISEDBH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECRK6LI#issuecomment-547532589, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACXCXWLBA5M4BJ4QCAG5GX3QRBVN3ANCNFSM4ISEDBHQ .

-- Max Kovalenko MileZero http://www.milezero.com/ | 206.349.0871 <//206.349.0871> 615 2nd Ave, Ste 150, Seattle, WA 98104

[github-actions[bot]]

This issue is marked stale. It will be closed in 30 days if it is not updated.