redis-om-python icon indicating copy to clipboard operation
redis-om-python copied to clipboard

Feature Request - lazy redis connection for JsonModel/HashModel

Open svabra opened this issue 1 year ago • 1 comments

ARCHITECTURE ANTI-PATTERN Declaring a datasource with the model is tightly coupling the two. Here an example of the Meta class added to a JsonModel/HashModel

REDIS_DATA_URL = os.environ.get("REDIS_OM_URL", 'redis://localhost:6380')
[..]
class Meta:
        global_key_prefix="performance-validation"
        model_key_prefix="order"        
        database = get_redis_connection(url=REDIS_DATA_URL, decode_responses=True)

ARGUMENT There is one dominant use cases when this becomes a problem at runtime in particular. You are using one or several JsonModels/HashModels in an application, you start the application but the redis server is not yet there or not yet ready. The application:

  • startsup ...
  • the imports are loaded (also the JsonModels/HashModels one wants to use)
  • the Models will try immediately to connect to redis and fail

The redis.Redis([..]) connection attemp can be handled with a @retry i.e. using the tenacity library. However, NOT the models. So the models will actually fail the applications. That CANNOT be a desired outcome. In distributed computing there is always the possibility that a service (like redis) is not available for whatever reason.

Furthermore, this also means, one cannot run github actions (or any other CI/CD test) unless one pulls up a redis service before testing. Yes, feasible, but not desirable.

SUGGESTION

  • invoke the connection attempt when one uses the model (e.g. before the save() method is invoked). Add a method connect
  • if there really is an advantage of a immediate connection at import-time, then we could implement eager and lazy-connection attempts.

(I do understand one can still use the Redis/StrictRedis object to "manually" add record to any of the redis instances. Anyhow, the redis-om-pyhton library is really the way to go. A very neat piece of software so far. Thanks to all who contribute to this library.)

REFERENCES Apparently I'm not the only one disliking the tight-coupling: https://github.com/redis/redis-om-python/issues/519

svabra avatar Jun 17 '23 07:06 svabra

Hi!

I'd like to add a (common?) use-case for this. We use a cache as a 'nice to have' and would like it to silently failover to doing nothing. In other words, if there is a ConnectionError, just treat all .get() by raising NotFound. The application continues, just without caching. This concept is well-described here (Django context, but same idea).

I've done a proof of concept for my purposes, which might interest some.

brettbeeson avatar Jul 16 '23 04:07 brettbeeson