socket.io-redis-adapter icon indicating copy to clipboard operation
socket.io-redis-adapter copied to clipboard

Error: 23984 trailing bytes

Open shivashahu07 opened this issue 1 year ago • 5 comments

"socket.io": "^4.5.4" "@socket.io/redis-adapter": "^8.0.0", AWS elastic cache - redis engine version - 6.2.6

Above are the version in my application running on mutliple container and i'm getting this error when again and again and below are the exception are coming from nowhere and I don't know the exact root cause:

Error: 23984 trailing bytes at Object.decode (/app/node_modules/notepack.io/lib/decode.js:289:11) at RedisAdapter.onmessage (/app/node_modules/@socket.io/redis-adapter/dist/index.js:131:34) at /app/node_modules/@socket.io/redis-adapter/dist/index.js:84:22 at PubSub._PubSub_emitPubSubMessage (/app/node_modules/@redis/client/dist/lib/client/pub-sub.js:294:9) at PubSub.handleMessageReply (/app/node_modules/@redis/client/dist/lib/client/pub-sub.js:237:93) at Object.onReply (/app/node_modules/@redis/client/dist/lib/client/commands-queue.js:40:87) at RESP2Decoder.write (/app/node_modules/@redis/client/dist/lib/client/RESP2/decoder.js:119:26) at RedisCommandsQueue.onReplyChunk (/app/node_modules/@redis/client/dist/lib/client/commands-queue.js:154:72) at RedisSocket.<anonymous> (/app/node_modules/@redis/client/dist/lib/client/index.js:410:84) at RedisSocket.emit (node:events:513:28) at Socket.<anonymous> (/app/node_modules/@redis/client/dist/lib/client/socket.js:201:42)

After this error my application got socket connection timeout reached issue and This issue is coming when i'm hitting 500+ connection all together:

Error: timeout reached while waiting for fetchSockets response at Timeout._onTimeout (/app/node_modules/@socket.io/redis-adapter/dist/index.js:568:28) at listOnTimeout (node:internal/timers:564:17) at process.processTimers (node:internal/timers:507:7)

Can anyone help why this is happening again and again.

shivashahu07 avatar Dec 09 '23 11:12 shivashahu07

Hi!

Error: xxx trailing bytes means the adapter has received an ill-defined packet from Redis PUB/SUB. Not sure how that could happen though.

timeout reached while waiting for fetchSockets() means the fetchSockets() operation has timed out because the other servers did not respond in the given delay. You might want to increase the requestsTimeout option:

const io = new Server({
  adapter: createAdapter(pubClient, subClient, {
    requestsTimeout: 10000
  })
});

Or simply retry upon timeout.

darrachequesne avatar Dec 11 '23 04:12 darrachequesne

@darrachequesne : I already did the requestTimeout to 10000 ms. but fetchSocket() is timedout since yesterdayand still giving the same error and my server cpu and memory utilization is 2-3%. Anything that redis are storing while pub/sub that are not cleared out within timeframe?

Whats your thought whether this is socket server issue or redis-adaptor issue?

shivashahu07 avatar Dec 11 '23 04:12 shivashahu07

@shivashahu07 its prbly caused by this

https://github.com/socketio/socket.io-redis-adapter/issues/367#issuecomment-1783669520

cody-evaluate avatar Dec 19 '23 17:12 cody-evaluate

@cody-evaluate : So how did you resolve this issue?

shivashahu07 avatar Dec 19 '23 17:12 shivashahu07

@shivashahu07 nope, happens intermittently but we will be moving to the sharded adapter soon.

cody-evaluate avatar Dec 21 '23 19:12 cody-evaluate