StackExchange.Redis icon indicating copy to clipboard operation
StackExchange.Redis copied to clipboard

Timeout awaiting response

Open MykolaKushnarenko opened this issue 1 year ago • 3 comments

Hey team!

Now and then we encounter "Timeout awaiting response" We're hosting our env on Azure and using Azure Cache for Redis Tier: Standard 1 GB

Here is the error message: Timeout awaiting response (outbound=0KiB, inbound=0KiB, 5884ms elapsed, timeout is 5000ms), command=GET, next: GET #, inst: 0, qu: 0, qs: 0, aw: False, bw: SpinningDown, rs: ReadAsync, ws: Idle, in: 0, last-in: 210, cur-in: 0, sync-ops: 0, async-ops: 18238, serverEndpoint: #, conn-sec: 22769.29, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: client(SE.Redis-v2.7.20.59961), IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=1,Free=32766,Min=1,Max=32767), POOL: (Threads=2,QueuedItems=0,CompletedItems=460840,Timers=10), v: 2.7.20.59961 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

The issue occurred at 23:49:36.5374749

Server load: image CPU: image Cache hit: image Cache read: image Total keys: image

Would it be at all possible to lend a helping hand? We would highly appreciate your help. Thanks in advance!

MykolaKushnarenko avatar Mar 14 '24 09:03 MykolaKushnarenko

This looks like the client is actually awaiting data. The first question that comes to mind is: how large are these values that the GET is fetching?

NickCraver avatar Mar 16 '24 15:03 NickCraver

Hey @NickCraver I'm really grateful for your reply!

There are basically 2 scenarios when this issue is reproduced:

  1. GET for a health check (GET HEALTHCHECK).
  2. GET to fetch some stashed data.

In the first case, we're expecting just a response (nil) even though there is nothing stored by this key(Key: HEALTHCHECK). But still, we get a timeout from time to time. SLOWLOG GET showed "GET HealthChech": image

Regarding real data, the size of the value was about 4.5 KB, a simple array of strings. It doesn't, however, occur all the time. It may happen a couple of times through 2 weeks period. SLOWLOG GET showed: image

MykolaKushnarenko avatar Mar 18 '24 11:03 MykolaKushnarenko

@NickCraver any ideas regarding the topic, root cause and solution?

bastyuchenko avatar Sep 19 '24 06:09 bastyuchenko