redis
Lettuce - Automatic connection retry on startup
lettuce has fantastic functionality for handling any Redis disconnections once everything is up and running. If the Redis instance becomes unavailable, use of an appropriate retry_strategy means the client can be set up to try to reconnect until Redis is available again.
Is it possible to get the client to enter this state at start-up, even if Redis is down?
My scenario is this: I am using Redis as a primary data-store, with a fallback secondary data-store which is not Redis-based. When my application starts, if Redis is not available, attempts to retrieve data will use the secondary data-store instead.
However, when Redis does become available, I'd like my application to start using the Redis primary data-store instead. As the Redis connection did not succeed on start-up, the retry_strategy, which would handle this for a previously established connection, does not work.
I could write code which will retry the initial Redis connection until it is successful, but it strikes me that the functionality available out-of-the-box is already very close to what I need, if I can persuade it to come into play from start-up even though Redis is down.
Pretty much copied from here: https://stackoverflow.com/questions/45836875/establish-connection-retry-behaviour-on-start-up-even-when-redis-is-not-availa
That isn't something we can really do. We have several assumptions that we need to fulfill at startup. For instance, with Redis Cluster, we must have the topology to be able to route commands to appropriate nodes.
I rather suggest using a connection pool to introduce fail safety. Asking the pool for a resource and if it succeeds, then perform Redis activity, otherwise, back off and follow the recovery code path.
We once had a similar ticked but I wasn't able to find it amongst our tickets.
Thank you very much for your answer.
I rather suggest using a connection pool to introduce fail safety. Asking the pool for a resource and if it succeeds, then perform Redis activity, otherwise, back off and follow the recovery code path.
Can that be used at startup? I mean, could a connection pool as it works today, be used to solve the describe problem, where I could create the pool at start-up even if redis is unavailable?
How does the connection pool feature work with the auto-reconnect and buffering of commands (which by the way are two amazing features), does the connection pool support these two features as well?
I mean, it seems a little silly to me to have to be dependent on redis. Many people only use it as a cache, I don't want to be dependent on my cache, in that if my cache doesn't work, well, I wouldn't want everything else to be failing just because of it. If redis is required at the beginning of my app, then my app is effectively dependent on redis, and this is especially a problem in today's world, where apps are often running on short-lived computers like Spot instances, so new apps have to start-up all the time.
I mean, I'm not trying to be disrespectful, I'm very grateful for lettuce, it's awesome, I would just like to objectively point out the flaw.
Can that be used at startup? I mean
Yes, if you do not set a min-idle size.
I wouldn't want everything else to be failing
I think this is pretty much an implementation detail. Clearly, acquiring a connection and remaining fail-safe introduces some complexity, yet a driver doesn't cater for all eventualities.
How does the connection pool feature work with the auto-reconnect and buffering of commands
Basically no different than using standalone connections. You request a connection from the pool, run your activity and return the connection back to the pool.
Using a pool has the advantage that you create an object and you can request connections from there. In contrast, if your application would use a single connection at startup, you would need to carry the connection (or a wrapper) across your application resulting in much more entanglement and complexity.
I would just like to objectively point out the flaw.
Failing on the initial connection retrieval is much safer for applications than silently handing out a connection that isn't connected. Starting in disconnected mode bears quite a moment of surprise because it isn't obvious and goes against many expectations.
Failing on the initial connection retrieval is much safer for applications than silently handing out a connection that isn't connected. Starting in disconnected mode bears quite a moment of surprise because it isn't obvious and goes against many expectations.
For some users, I would assume that that would be totally true. Other users, however, as I explained, don't want to be dependent on their cache. As a result, there are two different use cases, and the ability to support both could be obviously beneficial.
I'm not telling you to create this feature. I am definitely not in a position to do that, while using your software for free. I'm just trying to argue the counter side mainly for the sake of improving the software and possibly users who use it (me included).
You don't need to justify your decisions to me, but I hope you don't mind my assertiveness nonetheless.
Thank you for all your explanations concerning the connection pool feature. I will definitely considering looking into it. The reason I haven't yet, which I tried, but I discovered that the downstream library that I'm using doesn't support it.
Using a pool has the advantage that you create an object and you can request connections from there. In contrast, if your application would use a single connection at startup, you would need to carry the connection (or a wrapper) across your application resulting in much more entanglement and complexity.
This doesn't seem to be true to me. If I don't pass a single connection around, then I would have to instead pass a connection pool which is not any less complex or causes any more entanglement than a single connection would. And on the contrary, with connection pool, to draw a connection, use it, then return the connection, causes a lot more complexity than a single connection.
Furthermore, because redis is mostly just a key-value store, its queries are usually very fast, so a need for a connection pool is usually superficial.
I am concerned whether the connection pool could really 100% satisfy this requirement and not come with any downfalls at the same time. But I admit, I don't know, and yet I don't think I'll get around to looking into it in the near future, because I'm dependent on the downstream redis4cats library, which doesn't support connection pools anyways. Of course, feel free to close the issue.
Thank you.