Refactoring Hedis's Connection Pool Management For Clustered Redis

[ishan-juspay]

Currently for Clustered Redis connection pool management, we treat each resource of the pool as a list of active connection to each node of the cluster. This has the following problems:

1. If application is unable to connect to any one of the node (returned from Cluster Slots), client will not be able to create any new resource in the pool.
2. If one of the node in the cluster is having high latency, it might end up blocking all resources from the pool, thus causing the issue to spillover for the entire application. Ideally the issue should be confined to the operations happening to that specific node and operations to other well performing nodes shouldn't be impacted.

Idea is to explore if there should be a connection pool per node of the Redis cluster to solve this?

[qnikst]

I'm personally not a huge fan of hiding connection pool details into the package and not giving an access.

While it has some benefits like automatic reconnection using Redis-cluster or sentinel protocols, it does not give a user ability to express all the requirements of the particular system and complex algorithms. In our cases requirements were to have a back pressure algorithms to avoid cascading failures. So I'd rather see connection exposed and possibly some library helpers that implements most common scenarios.

However I never used Redis-cluster and at work we are using sentinel so I may miss particular.

So I really would trust the approach suggested by cluster users, and as far as I understand Juspay developers are such users.