ConnectionTimeoutError: Connection timeout

[Thalari-Kolarivan]

trafficstars

Description

Code snippet for connecting the redis is below.

    this.client = redis.createClient({
      socket: { host: redisHost, port: REDISPORT},
    });
    this.client.connect();
    this.client.on("error", (err) => log.error("Error in redis :", err));

The error we got in our logs is below.

    ConnectionTimeoutError: Connection timeout
    at Socket.<anonymous> (/workspace/node_modules/@redis/client/dist/lib/client/socket.js:178:124)
    at Object.onceWrapper (node:events:627:28)
    at Socket.emit (node:events:513:28)
    at Socket.emit (node:domain:489:12)
    at Socket._onTimeout (node:net:550:8)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)

We also tried configuring connectTimeout for 10 seconds in the createClient after that we didn't get this error. Should we have to configure the timeout ? Can we have a proper solution or a work around for this Connection Timeout Error?

Node.js Version

v16.19.1

Redis Server Version

6.2.7

Node Redis Version

4.6.7

Platform

Google appengine

Logs

No response

[rajat42059]

• 1 timeout error

/node_modules/ioredis/built/Redis.js:170:41) at Object.onceWrapper (node:events:627:28) at Socket.emit (node:events:513:28) at Socket.emit (node:domain:489:12) at Socket._onTimeout (node:net:568:8) at listOnTimeout (node:internal/timers:564:17) at processTimers (node:internal/timers:507:7) { errorno: 'ETIMEDOUT', code: 'ETIMEDOUT', syscall: 'connect'

[rajat42059]

Any Update how to fix?

[leibale]

Usually, the cause of timeout errors on start-up is a blocked event loop (the app start > the client start to connect > the event loop is blocked for more than connectTimeout > the connection throws the timeout error). You can use the blocked-at package to help you understand what is blocking.

I would not suggest you to use a high number for connectTimeout, since it used for reconnects as well.

[rajat42059]

const client = createClient({ url: 'redis://127.0.0.1:6379', socket: { connectTimeout:5000 },

This is also givng same timeout error,

[leibale]

@rajat42059 from the error stack in this comment it looks like you are using ioredis... (this repo is for node-redis)

have you tried connecting to your redis server using redis-cli?

[rajat42059]

I am using this https://www.npmjs.com/package/redis

also I am able to connect vis reis 3.1.2 but not with latest version

[Thalari-Kolarivan]

@leibale I have added blockedAt in our application. I have removed the connectTimeout. But it is not happening now. Is there is any steps to reproduce the same issue ?

Added the below code to detect the slow execution by refering blockedAt documentation.

blocked((time, stack, {type, resource}) => {
 console.log(`Blocked for ${time}ms, operation started here:`, stack)
 if (type === 'HTTPPARSER' && resource) {
   // resource structure in this example assumes Node 10.x
   console.log(`URL related to blocking operation: ${resource.resource.incoming.url}`)
 }
}, {resourcesCap: 100})

[leibale]

something like:

const timeout = 1000,
  client = createClient({
    socket: {
      connectTimeout: timeout
    }
  });

client.on('error', err => console.error(err));

client.connect().catch(err => console.error(err));
const start = Date.now();
while (Date.now() - start < timeout) {} // block the event loop

[leibale]

@Thalari-Kolarivan did you manage to fix it?

[Shraddha-Dabholkar]

Any fix for this?

[Thalari-Kolarivan]

@leibale Sorry, I took more time to get back. We are using gcp redis 6.x , it got auto-upgraded to 6.2.11 . After that we are not facing the same error. We are observing the logs.

[dermasmid]

In my case it was cpu starvation because we removed cpu limits from our k8s workloads

[rajat42059]

for now I am using redis version 3.5.1 which seems to be working fine for me

[essential-randomness]

Can confirm I'm also having the same problem connecting to redis from google cloud/appengine with version 4. Upgrading to redis 6.x didn't work, but downgrading the library seems to do the trick.

[im-znd]

Any fix for this?