RedisConnectionException when using DistributedLock.Redis

[steingran]

Hi, we just started using DistributedLock.Redis, and on my developer machine it works great. But in our development environment on AWS we get this one pretty frequently (9300 times in 3 days):

StackExchange.Redis.RedisConnectionException: UnableToConnect (None, 0-read, last-recv: 0) on dev-ec-valkey-abc.serverless.eun1.cache.amazonaws.com:6379/Interactive, Flushed/ReadAsync, last: ECHO, origin: ResetNonConnected, outstanding: 12, last-read: 5s ago, last-write: 5s ago, unanswered-write: 5s ago, keep-alive: 60s, state: ConnectedEstablishing, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.7.33.41805 StackExchange.Redis.RedisConnectionException: The message timed out in the backlog attempting to send because no connection became available (5000ms) - Last Connection Exception: UnableToConnect (None, 0-read, last-recv: 0) on dev-ec-valkey-abc.serverless.eun1.cache.amazonaws.com:6379/Interactive, Flushed/ReadAsync, last: ECHO, origin: ResetNonConnected, outstanding: 12, last-read: 5s ago, last-write: 5s ago, unanswered-write: 5s ago, keep-alive: 60s, state: ConnectedEstablishing, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.7.33.41805, command=EVAL, timeout: 5000, inst: 0, qu: 2, qs: 12, aw: False, bw: CheckingForTimeout, rs: ReadAsync, ws: Idle, in: 0, in-pipe: 0, out-pipe: 0, last-in: 0, cur-in: 0, sync-ops: 0, async-ops: 9062, serverEndpoint: dev-ec-valkey-abc.serverless.eun1.cache.amazonaws.com:6379, conn-sec: 20, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: ip-10-24-0-62(SE.Redis-v2.7.33.41805), IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=1,Free=32766,Min=2,Max=32767), POOL: (Threads=4,QueuedItems=0,CompletedItems=2036620,Timers=22), v: 2.7.33.41805 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts) at async Task<RedisResult> RedisDatabase.ScriptEvaluateAsync(string script, RedisKey[] keys, RedisValue[] values, CommandFlags flags)() in //src/StackExchange.Redis/RedisDatabase.cs:line 1551 at async Task RedLockHelper.AsBooleanTask(Task<RedisResult> redisResultTask)() in //src/DistributedLock.Redis/RedLock/RedLockHelper.cs:line 63 System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread.

Note that we use Valkey, not Redis.

Our code looks like this:

var semaphore = new RedisDistributedSemaphore(semaphoreKey, MaxConcurrentRequests, redis.GetDatabase());
IDistributedSynchronizationHandle? handle = await semaphore.TryAcquireAsync(TimeSpan.FromSeconds(SemaphoreTimeoutSeconds), cancellationToken);

Where the 'redis' variable is of type IConnectionMultiplexer.

What could be the reason for these exceptions?

[madelson]

Seems like you're hitting a timeout. Some ideas:

• Try reading the article linked in the error message and see if anything applies to your setup
• Try both the latest version (1.1.1) and the previous version (1.1.0). Does either work?
• Try issuing a Redis command on the connection prior to using the lock. Does that time out?