vertx-redis-client
vertx-redis-client copied to clipboard
vert-x3
Reading from read replicas when using Elasticache (cluster mode disabled)
Describe the feature
Currently, when setting the client to 'REPLICATION' mode, it performs an INFO command to 'master' and extracts its 'slaves' IPs from the reply. Using this with AWS Elasticache (cluster mode disabled) is faulty since the returned IPs are internal. This results in connection failures with the replicas leaving only communication with 'master'. This is not scalable and is not leveraging replication.
Using AWS Elasticache requires explicit configuration of the "Read endpoint" for leveraging the read replicas (see here).
The client should allow this configuration as an alternative for auto-discovering the replicas IPs.
Use cases
This feature will enable proper usage of vert.x redis client with AWS Elasticache (cluster mode disabled) which seems very important. In addition, this will benefit other Redis based servers which also hide their replicas IPs behind NATs
This can be easily reproduced by configuring the vert.x redis client to 'REPLICATION' mode and setting it to work with a basic Elasticache setup (cluster mode disabled) with 2 read replicas (configuring the endpoint to Elasticache 'primary' endpoint).
This is a know issue of AWS Elasticache that most of drivers need to take care of. I took a quick look how other mature well know drivers cope with this and found that jedis does not have mechanism to support abstraction of AWS Elasticache and Lettuce has some mechanism that can be used in order to use Redis replication to full potential via AWS Elasticache.
What can be done from user side is that you can create 2 clients. One using AWS primary endpoint and you can execute write commands on this client. Also you can create a second client with address of reader endpoint to take care of read commands. You just need to make a decision when issuing commands if it is read or write command in order to use the right client.
Feature inspired by Lettuce can be introduced in Vert.x where we can set a nondynamic/static setup up nodes with their role so client can then use this info to perform commands on the correct endpoint. It all depends if we want this kind of abstraction above Redis client especially as it is use case specific for AWS Elasticache. More on the solution Static Master/Replica with predefined node addresses
It should be fairly simple to add a pair of method like RedisOptions.setStaticMaster() and RedisOptions.setStaticReplicas(). If they are called, the provided values would be used instead of automatic discovery from the configured endpoints.
For Elasticache, if I understand correctly, there would be just one replica (there's load balancing behind the scenes), but it seems best to allow configuring an entire list of replicas.
I'm thinking this would currently only apply to the REPLICATION mode, though it could possibly be extended to CLUSTER and SENTINEL in the future, if there's demand and reasonable semantics.
I have an implementation (in this branch https://github.com/Ladicek/vertx-redis-client/commits/static-topology/) that allows configuring topology statically for the replication client. In the end, I decided to just add a flag whether current configuration statically defines the topology (RedisTopology.STATIC) and if so, the first endpoint is treated as the master and the rest as replicas. I'll be able to submit a PR once #428 / #429 are merged.
