When a node is terminated in AWS EKS, ioredis used in a nodejs app is giving "Error: Connection is closed"

[ramprasad1996]

Hi,

We are using a node js based app that uses ioredis for connecting to Redis. The app is hosted inside AWS EKS cluster. We observed an issue wherein the node which hosts the Redis pod is terminated, the ioredis continuously gives "Error: Connection is closed" error response and is not able to connect back to Redis. There are 6 replica set for the app in the cluster. When the failover occurs, this issue is happening only for 2-3 pods everytime. Other pods were able to detect the failover and connect back successfully.

We are using Redis sentinel-based setup for HA.

redis_url: [ 'redis://redis-0.redis..svc.cluster.local:6379', 'redis://redis-1.redis..svc.cluster.local:6379', 'redis://redis-2.redis..svc.cluster.local:6379' ], redis_master: 'redis-master', redis_sentinel_port: 26379, redis_password: '[REDACTED]', redis_max_retries: '3', redis_command_timeout: '5000', redis_sentinel_reconnect_delay: '60000', redis_retry_delay: '5000', redis_max_reconnect: '15', is_canary: false, redis_standalone_mode: false, redis_autopipelining: 'true' }

Error logs : Error when trying to read data from cache Error: Connection is closed. at Redis.sendCommand (/snapshot/mnt/node_modules/ioredis/built/redis/index.js:628:24) at execPipeline (/snapshot/mnt/node_modules/ioredis/built/pipeline.js:373:25) at Pipeline.exec (/snapshot/mnt/node_modules/ioredis/built/pipeline.js:285:16) at Immediate.executeAutoPipeline [as _onImmediate] (/snapshot/mnt/node_modules/ioredis/built/autoPipelining.js:46:14) at processImmediate (internal/timers.js:463:21) { [stack]: 'Error: Connection is closed.\n' + ' at Redis.sendCommand (/snapshot/mnt/node_modules/ioredis/built/redis/index.js:628:24)\n' + ' at execPipeline (/snapshot/mnt/node_modules/ioredis/built/pipeline.js:373:25)\n' + ' at Pipeline.exec (/snapshot/mnt/node_modules/ioredis/built/pipeline.js:285:16)\n' + ' at Immediate.executeAutoPipeline [as _onImmediate] (/snapshot/mnt/node_modules/ioredis/built/autoPipelining.js:46:14)\n' + ' at processImmediate (internal/timers.js:463:21)', [message]: 'Connection is closed.'

ioredis version : 4.28.0 redis version : 6.2.7

Please help on this issue.

Thanks, Ram

@luin

[TxusBlack]

Same here. Version ^5.3.2