Support for Redis Cluster via Carmine Library

[olttwa]

Context

• In past, we've struggled with closing connection pool during graceful shutdown, lack of Redis v6.2.0 support.
• Despite these struggles, after spiking various libraries (link to Redis ADR), we decided to stick with Carmine.

Problem

• Carmine lacks support for Redis Cluster

Solution

• Goose can have a small set of macros for the subset of Redis operations to map keys to the right cluster node. This macro will rewrite Redis calls to follow the cluster protocol of identifying the node and then query it
• If these macros can be generalised, we can raise a PR to Carmine itself

[olttwa]

Thanks for bringing this to our notice @jaju @kapilreddy If you have code/better solution for this, do suggest here

[fr33m0nk]

@olttwa You may want to look into RedClaw (Clojure wrapper over Redisson) or Celtuce (Clojure wrapper over Lettuce) or Obiwan (Clojure wrapper over Jedis) All three support Redis cluster natively. RedClaw and Celtuce also support asynchronous event driven mode of execution unlike Jedis.

[tfidfwastaken]

A less drastic solution to replacing the Redis library (as noted in the ADR, most of the other libraries are sorely lacking some commands and features that would also need to be implemented) is to write our own version of the wcar macro that returns a handle to the connection. The functions needed to create our own wcar+ are exposed to the user.

Moreover, Carmine will support Redis Cluster in the next release—it would also be a good time to address the issues with closing connections after we upgrade. So currently, I am of the opinion that we should wait on this issue a bit until Carmine 4.0 is out, and then make a decision.