Turn the caching system into an interface to support different caching strategies

[efirvida]

I thought it would be a good idea to have several caching strategies, so I modified a bit the cache API to give ideas of how it could be, for example, to use Redis as a caching system.

import redis
from beanie import Document
from beanie.odm.cache import RedisCache, LRUCache

redis_client = redis.Redis(host='localhost', port=6379, db=0)

class Product(Document):
    name: str
    ...

    class Settings:
        use_cache = True
        cache_system = RedisCache(redis_client)

class User(Document):
    name: str
    ...

    class Settings:
        use_cache = True
        cache_system = LRUCache(
            capacity=32, expiration_time=timedelta(seconds=10)
        )

[roman-right]

Oh wow! Great! I'll check everything and merge it tomorrow. Thank you for this feature

[roman-right]

It looks like it breaks some tests

[efirvida]

It looks like it breaks some tests

Hi!, I'm, not surprised. This was just an idea to work on, that came to me on a project that I want to use Beanie. And also because I'm learning about Redis as a caching system.

I did not expect such a quick and positive reaction to my proposal. I'm going to work it better to make it ready for merge. Because even the RedisCashe implementation is very poor at this point.

[efirvida]

@roman-right, I have made some fixes and improved de Redis cache system with msgpack as serialization and add ttl.

But there is something strange that my first commit brake (f6614c4ccf4ed8613c70c1d23d174c4af6ef3f79) some test "no cache-related" from my point of view its the same thing but not for the test.

if I change this:

  @classmethod
  def init_cache(cls) -> None:
      """
      Init model's cache
        :return: None
        """
        if cls.get_settings().use_cache:
            cls._cache = cls.get_settings().cache_system

for this:

    @classmethod
    def init_cache(cls) -> None:
        """
        Init model's cache
        :return: None
        """
        if cls.get_settings().use_cache:
            cls._cache = LRUCache(capacity=5, expiration_time=datetime.timedelta(seconds=10))

Which I think is the same thing all test runs, but not if I init the cache from the Document's cache_system settings

[efirvida]

0c9d28e

Fixed here 0c9d28e

[efirvida]

Now it's working, here is an example code

import redis
import msgpack
import json
import pickle
from typing import Optional
from datetime import timedelta

from beanie.odm.cache import LRUCache,RedisCache

redis_client = redis.Redis(host='localhost', port=6379, db=0)

class ProductNoCache(Document):
    name: str
    description: Optional[str] = None
    price: Indexed( float ) 
    class Settings:
        use_cache = False


class ProductRedisPickle(Product):
    class Settings:
        use_cache = True
        cache_system = RedisCache(redis_client, serializer=pickle) # pickle is default serializer


class ProductRedisJSON(Product):
    class Settings:
        use_cache = True
        cache_system = RedisCache(redis_client, serializer=json, ttl=timedelta(seconds=10)) # use ttl for expiration
        
       
class ProductRedisMsgPack(Product):
    class Settings:
        use_cache = True
        cache_system = RedisCache(redis_client, serializer=msgpack, ttl=10) # ttl can be int -> 10 seconds


class ProductLRUCache(Product):
    class Settings:
        use_cache = True
        cache_system = LRUCache(capacity=5, expiration_time=timedelta(seconds=10))```

[roman-right]

Wow! You worked a lot here. Thank you! I'll check everything this weekend.

[roman-right]

Hey! I've reviewed and found a few issues. If you have time, it would be great to fix it. If not, please let me know.

[github-actions[bot]]

This PR is stale because it has been open 45 days with no activity.

[github-actions[bot]]

This PR was closed because it has been stalled for 14 days with no activity.