Redis cache: `get()` does not work for incremented values (returns '_pickle.UnpicklingError: could not find MARK')

[simon-liebehenschel]

Steps to reproduce:

await cache.increment("foobareggs", delta=10)
assert await cache.get("foobareggs")

Result:

>       assert await cache.get("foobareggs")

test_common_cache.py:40: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../src/common/cache.py:230: in get
    return await self.cache.get(key, default, loads_fn=loads_fn, namespace=namespace or self.namespace, timeout=timeout or self.timeout, **kwargs)
../../venv/lib/python3.9/site-packages/aiocache/base.py:61: in _enabled
    return await func(*args, **kwargs)
../../venv/lib/python3.9/site-packages/aiocache/base.py:45: in _timeout
    return await asyncio.wait_for(func(self, *args, **kwargs), timeout)
/usr/lib/python3.9/asyncio/tasks.py:481: in wait_for
    return fut.result()
../../venv/lib/python3.9/site-packages/aiocache/base.py:75: in _plugins
    ret = await func(self, *args, **kwargs)
../../venv/lib/python3.9/site-packages/aiocache/base.py:192: in get
    value = loads(await self._get(ns_key, encoding=self.serializer.encoding, _conn=_conn))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <aiocache.serializers.serializers.PickleSerializer object at 0x7f5042157160>
value = b'10'

    def loads(self, value):
        """
        Deserialize value using ``pickle.loads``.
    
        :param value: bytes
        :returns: obj
        """
        if value is None:
            return None
>       return pickle.loads(value)
E       _pickle.UnpicklingError: could not find MARK

../../venv/lib/python3.9/site-packages/aiocache/serializers/serializers.py:119: UnpicklingError

Update:

It fails only with the PickleSerializer, but workds with the JsonSerializer. Is there any way to make this working without switching a serializer?

[citramon]

In this case it doesn't work with JsonSerializer too:

from aiocache.serializers import JsonSerializer
cache = Cache(Cache.MEMORY, serializer=JsonSerializer())
loop.run_until_complete(cache.set("a", 1))
True
loop.run_until_complete(cache.get("a"))
1
loop.run_until_complete(cache.increment("a"))
2
loop.run_until_complete(cache.get("a"))
Traceback (most recent call last):
  File "/Users/sbushuev/.pyenv/versions/3.7.9/lib/python3.7/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/Users/sbushuev/.pyenv/versions/3.7.9/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    return future.result()
  File "/Users/sbushuev/projects/scream-team/cat-event-collector/env/lib/python3.7/site-packages/aiocache/base.py", line 61, in _enabled
    return await func(*args, **kwargs)
  File "/Users/sbushuev/projects/scream-team/cat-event-collector/env/lib/python3.7/site-packages/aiocache/base.py", line 45, in _timeout
    return await asyncio.wait_for(func(self, *args, **kwargs), timeout)
  File "/Users/sbushuev/.pyenv/versions/3.7.9/lib/python3.7/asyncio/tasks.py", line 442, in wait_for
    return fut.result()
  File "/Users/sbushuev/projects/scream-team/cat-event-collector/env/lib/python3.7/site-packages/aiocache/base.py", line 75, in _plugins
    ret = await func(self, *args, **kwargs)
  File "/Users/sbushuev/projects/scream-team/cat-event-collector/env/lib/python3.7/site-packages/aiocache/base.py", line 192, in get
    value = loads(await self._get(ns_key, encoding=self.serializer.encoding, _conn=_conn))
  File "/Users/sbushuev/projects/scream-team/cat-event-collector/env/lib/python3.7/site-packages/aiocache/serializers/serializers.py", line 151, in loads
    return json.loads(value)
  File "/Users/sbushuev/.pyenv/versions/3.7.9/lib/python3.7/json/__init__.py", line 341, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not int

[Dreamsorcerer]

It seems to me like increment() is meant to be an efficient, atomic operation on the backend. If the value is serialised to anything other than an int then this is obviously not possible.

So, I'd suggest not using increment() if you serialise to something that is not an int stored in the backend. Instead just do set(k, get(k) + 1) or similar as if the increment() method didn't exist.

[Dreamsorcerer]

Duplicate of #482.