Conditions for numeric comparisons (counters)

[kmcclellan]

Looking at the supported kinds of Condition for use with transactions, I was surprised there is not a way to compare two Redis values numerically (greater than, less than, etc.). The Redis "string" type is often used to represent numeric counters (as indicated by the INCR/DECR commands).

[mgravell]

You're not wrong, and it would be an interesting addition. However, my strong recommendation would be to use Lua for this instead. As it happens, I had reason to do a performance experiment on condition-based transactions vs Lua last week, and the different was incredible, even on a localhost (minimising the latency cost) and without even considering the impact to unrelated concurrent pipeline usage.

[kmcclellan]

@mgravell That's interesting, I wonder what accounts for the difference. I assumed that the way the multiplexer works the commands in a transaction typically get batched and sent in a single round trip. Is your theory that Lua executes faster on the server for some reason?

If the difference were not substantial, I would likely still opt to express logic in .NET as opposed to Lua. There are advantages to maintainability/testability. There's also the drawback you mention in the documentation of Lua monopolizing the server.

As long as transactions are still a supported part of the product, I think there's a case for improving the available kinds of Condition. Admittedly low priority; your roadmap already has a lot of good improvements coming our way!

[mgravell]

A transaction with a condition is at least 5 operations:

WATCH
GET (or similar)
MULTI
(some operation)
(either EXEC or ABORT)

The library can issue everything up to the last step in one chunk, but it needs to see the outcome of the GET (etc) in order to decide whether to EXEC or ABORT, so that adds:

• a bunch of additional message bookkeeping
• an enforced latency penalty in the middle of the core IO loop
• ...during which time nobody can use the connection, hence impact to unrelated callers

IIRC the difference was multiple orders of magnitude vs Lua, which was only a small factor slower than having a dedicated command (my experiment was with INCR implemented via Lua vs directly)

[mgravell]

Found my numbers:

INCR: 1,000,000 in 1874ms 
EVAL: 1,000,000 in 3011ms
MULTI: 20,000 in 2183ms // note different number!!!

with test harness as below; note this is a little forced because we're trying to use F+F extensively to prove the point, but: transactions with constraints utterly destroy the pipeline.

var client = GetClient(0);
await client.StringSetAsync(CounterKey, "0").ConfigureAwait(false);
var watch = Stopwatch.StartNew();
for (int i = 0; i < OperationsPerClient; i++)
{
    await client.StringIncrementAsync(CounterKey, flags: CommandFlags.FireAndForget).ConfigureAwait(false);
}
var final = (int)await client.StringGetAsync(CounterKey).ConfigureAwait(false);
watch.Stop();
Console.WriteLine($"INCR: {final} in {watch.ElapsedMilliseconds}ms");
const string Incr = """
        local oldVal = redis.call('GET', KEYS[1])
        local newVal = tonumber(oldVal) + 1
        redis.call('SET', KEYS[1], newVal)
        return newVal
        """;
// prime the script to make sure we're using EVALSHA
var keysArray = new RedisKey[] { CounterKey };
await client.ScriptEvaluateAsync(Incr, keysArray).ConfigureAwait(false);
await client.StringSetAsync(CounterKey, "0").ConfigureAwait(false);
watch = Stopwatch.StartNew();
for (int i = 0; i < OperationsPerClient; i++)
{
    await client.ScriptEvaluateAsync(Incr, keysArray, flags: CommandFlags.FireAndForget).ConfigureAwait(false);
}
final = (int)await client.StringGetAsync(CounterKey).ConfigureAwait(false);
watch.Stop();
Console.WriteLine($"EVAL: {final} in {watch.ElapsedMilliseconds}ms");
await client.StringSetAsync(CounterKey, "0").ConfigureAwait(false);
watch = Stopwatch.StartNew();
for (int i = 0; i < OperationsPerClient; i++)
{
    var tran = ((IDatabase)client).CreateTransaction();
    tran.AddCondition(Condition.KeyExists(CounterKey)); // not really right, but: good enough
    _ = tran.StringIncrementAsync(CounterKey);
    await tran.ExecuteAsync(CommandFlags.FireAndForget).ConfigureAwait(false);
}
final = (int)await client.StringGetAsync(CounterKey).ConfigureAwait(false);
watch.Stop();
Console.WriteLine($"MULTI: {final} in {watch.ElapsedMilliseconds}ms");