ioredis icon indicating copy to clipboard operation
ioredis copied to clipboard

Receiving "Unhandled error event" with error handler

Open didomenicom opened this issue 3 years ago • 2 comments

In a similar situation as described in #896 and #969, I am receiving the following error.

ERROR	[ioredis] Unhandled error event: Error: connect ETIMEDOUT
    at TLSSocket.<anonymous> (/var/task/node_modules/ioredis/built/redis/index.js:317:37)
    at Object.onceWrapper (events.js:482:28)
    at TLSSocket.emit (events.js:376:20)
    at TLSSocket.emit (domain.js:470:12)
    at TLSSocket.Socket._onTimeout (net.js:483:8)
    at listOnTimeout (internal/timers.js:555:17)
    at processTimers (internal/timers.js:498:7)

My code looks like this

const redisClient = new Redis(`rediss://<username>:<pasword>@<host>:<port>`, {
	reconnectOnError(error: Error) {
		console.log(`[Redis]: Processing reconnectOnError - "${error.message}`)
		const targetErrors = [/READONLY/, /ETIMEDOUT/];

		targetErrors.forEach((targetError) => {
			if (targetError.test(error.message)) {
				console.log("[Redis]: reconnectOnError true")
				return true;
			}
		});
	},
})

redisClient.on('connect', () => {
  logger.debug(`Connecting to redis "${process.env.REDIS_ENDPOINT}:${process.env.REDIS_PORT}"...`)
})

redisClient.on('ready', () => {
  logger.debug('Connected to redis!')
})

redisClient.on('error', (err: Error) => {
  logger.error({
    message: 'Error with redis client',
    errorMessage: err.message,
    errorStack: err.stack,
    errorName: err.name
  })
})

redisClient.on('close', () => {
  logger.debug('[Redis]:closed')
})

redisClient.on('reconnecting', (time: Number) => {
  logger.debug(`[Redis]:reconnecting - ${time}ms`)
})

redisClient.on('end', () => {
  logger.debug('[Redis]:end')
})

redisClient.on('wait', () => {
  logger.debug('[Redis]:wait')
})

From what I can tell, this function has no event listeners defined (count == 0) and is falling into the console.error().

How do I setup the code so that my error handling code will catch this error and it will not hit the console.error() section of code in silentEmit()?

didomenicom avatar Jul 16 '21 12:07 didomenicom

Yes, I'm seeing this too, w/ sentinels and I wound up at the same place in the code.

In my case we are running sentinels in kubernetes and connecting to them directly. If a pod goes down, it'll get recreated, but the IP may have changed, so the connect to the sentinel will start emitting ETIMEDOUT errors, but the error handler on the main instance is never triggered.

const redis = new Redis({
  sentinels: ['host-1:26379', 'host-2:26379', 'host-3:26379']
, name: 'sentinel'
, enableReadyCheck: true
, lazyConnect: true
, enableOfflineQueue: true
, failoverDetector: true
, retryStrategy: () => {
    return RECONNECT_TIMEOUT
  }
, sentinelRetryStrategy: () => {
    return RECONNECT_TIMEOUT
  }
, reconnectOnError: (err) => {
    /* istanbul ignore if */
    if (err.message.includes('READONLY')) {
      log.warn({err}, 'readonly event triggered')
      return 2
    }
  }
})

redis.on('error', (err) => {
  process.nextTick(() => {
    throw err
  })
})

For a stand alone connection, I would expect the error handler here to be triggered on these timeout errors. For sentinels, I would expect the connection w/ error to be marked as dead or have some number of reconnection attempts. And the error handler should be tripped when all of the sentinels are unreachable. Timeout errors on connections should probably be handled similarly to unreachable errors.

esatterwhite avatar Jul 16 '21 18:07 esatterwhite

hello @esatterwhite can you please tell me what value are you using for RECONNECT_TIMEOUT ?

kodeine avatar May 09 '22 14:05 kodeine

How to catch connection timeout error?

truemoroz avatar Nov 18 '22 02:11 truemoroz