StackExchange.Redis icon indicating copy to clipboard operation
StackExchange.Redis copied to clipboard

Is there a way to execute individual commands with FailFast behavior?

Open madelson opened this issue 1 year ago • 3 comments

I have some logic using the RedLock algorithm to fire commands against multiple servers and recover if some servers are down. I recently updated from 2.2.4 to 2.7.17, and some of my tests are failing because the redis client now takes much longer to return when the server is disconnected. Switching the multiplexer to use BacklogPolicy.FailFast fixes this, but in general I want the new policy, just not for these particular operations where I want to fail quickly.

Is there a recommended way to achieve this? Should I be calling IDatabase.IsConnected() before every fail fast request and bailing if that returns false or is that going to hurt performance/have different semantics? If there's no way to achieve this, would it make sense to add a CommandFlags option to support it?

The way I simulate a downed server in my tests is as follows:

  • Launch a Redis process
  • Connect to it via a multiplexer
  • Grab an IDatabase instance off the multiplexer
  • Kill the Redis process with multiplexer.GetServer(...).Shutdown(ShutdownMode.Never);

Thanks for creating and maintaining this great library!

madelson avatar Feb 11 '24 14:02 madelson

Since we are a multiplexed pipeline, the backlog behavior can't really be per-operation since they're sharing a queue. Effectively, it's a behavior attribute of that overall queue (per connection). If you want to have different queue behavior for a certain set of things, I'd recommend a second multiplexer with that configuration since inherently they cannot share a config here, and use logic in the app side to decide which multiplexer to send the command(s) on.

NickCraver avatar Feb 11 '24 15:02 NickCraver

Thanks for the quick response @NickCraver .

What do you think about the option of calling IsConnected() before each request? What is the overhead of that like?

For context I'm working on a library where an IDatabase is passed to me, so I'd like to avoid requiring the caller to configure their connection in a particular way if possible.

madelson avatar Feb 12 '24 00:02 madelson

@madelson I'm not sure that makes a whole lot of sense when we think in pipeline terms. For example, it can disconnected immediately after your check - a flow of things is at some point interrupted and you won't know that immediately, there will always be some delay however small. The overhead is minimal - that's a cached bool on our side, and it would make some difference, but keep in mind that it can fail after the check is all.

The CommandFlags is an open question - curious what @mgravell thinks here. It is something we could add like CommandFlags.NoBacklog or something...though that name is accurate, I don't think it's intuitive. Perhaps CommandFlags.NoBacklogIfDisconnected or some such?

@mgravell thoughts on such a thing? To my knowledge, this if the first time the request has come up, but I could imagine many other cases to utilize such a thing.

NickCraver avatar Feb 17 '24 14:02 NickCraver