Is it possible to have simple cache interface?

[MasterSergius]

Hi, although lib is great finding, I'd like to not use that complex decorator method, but something like:

cache = AsyncCache()
cache.add(key, value)
cache.get(key)

And cache in this case is the single cache instance across whole async app.

[zmumi]

This library is focused on memoization and due to that, it has to follow different principles than put/get caches. Think more of a loading cache like in caffeine.

Let me share a couple of things that may be incompatible:

• Cache entries are characterized by timestamps governing when the cache should refresh/discard the entry - with get/put caches there is no refreshing or stale values.
• With get/put caches dogpiling is no longer an issue of the cache and application code should handle that instead (what renders the major advantage of this library unused).
• The library provides some form of request collapsing - this part also becomes unused (in the worst case it remains used and causes unnecessary operations).

I suppose we could use some building blocks of this library to create put/get cache but it could make the library more complicated (plus requires extra work).

Have you considered changing your caching pattern a little bit? If your flow looks like:

async def put_get_cache_interaction(cache: AsyncCache, key):
    cached = await cache.get(key)
    if cached is None:
        value = await expensive_computation(key)
        cache.put(key, value)

async def expensive_computation(key):
    return 'do-something'

It could be easily converted to something like:

async def expensive_computation(key):
    return 'do-something'

Alternatively (if dogpiling is not an issue) maybe you could take a look at aiocache? Methods like aiocache.caches.get or aiocache.caches.create could be useful.

[zmumi]

If you still want to use this lib but not to use decorators you could try a workaround (I'm sharing that with you but I haven't used that one myself yet properly)

import asyncio

from memoize.configuration import CacheConfiguration, DefaultInMemoryCacheConfiguration, MutableCacheConfiguration
from memoize.wrapper import memoize


class AsyncCache(object):

    def __init__(self, configuration: CacheConfiguration = DefaultInMemoryCacheConfiguration()) -> None:
        super().__init__()
        self.configuration = configuration
        wrapper = memoize(None, configuration)
        self.memoized = wrapper(self._load_stub)

    async def _load_stub(self, *args):
        raise ValueError("there is on loading in that cache")

    async def get(self, key):
        try:
            result = await self.memoized(self._load_stub, key)
            return result
        except CachedMethodFailedException:
            return None

    async def put(self, key, value):
        actual_key = self.configuration.key_extractor().format_key(self._load_stub, (self._load_stub, key,), {})
        entry = self.configuration.entry_builder().build(actual_key, value)
        await self.configuration.storage().offer(actual_key, entry)


async def main():
    # there is no point in configuring `set_method_timeout` as there is no loading
    # there is no point in configuring `key_extractor` as it is now implementation detail
    sample_config = MutableCacheConfiguration \
        .initialized_with(DefaultInMemoryCacheConfiguration()) \
        .set_entry_builder(ProvidedLifeSpanCacheEntryBuilder(
            # `update_after` = `expire_after` as there is no loading and each update effectively clears the key
            update_after=timedelta(milliseconds=100),
            expire_after=timedelta(milliseconds=100)
        )) \
        .set_eviction_strategy(LeastRecentlyUpdatedEvictionStrategy(capacity=256))

    cache = AsyncCache(configuration=sample_config)
    await cache.put('1', 'test1')
    await cache.put('3', 'test3')
    print(await cache.get('1'))
    print(await cache.get('2'))
    print(await cache.get('3'))


if __name__ == "__main__":
    asyncio.get_event_loop().run_until_complete(main())

The class itself does little and probably should work. But still, this lib was created to be a decorator for async methods that solves dogpiling and using this lib (as shown above) is not recommended due to inefficiency and neglecting dogpilling proofness.

[MasterSergius]

Thank you for the reply and good explanation. I need some control over the cache. In fact, I've started a library CacheControl for aiohttp few days ago. In theory, in single-threaded scenario my cache should avoid dog-piling problem. But if I can use some already existing solution, why not? What do you think, can it fit into my scenario? Also, I'm going to store cache values as dict, should be something like the following:

{'created_at_timestamp': <value>, 'max-age': <value>, 'some_value': <value>}

[zmumi]

As for having per-entry max-age I added an example to the docs on how to achieve that (probably too late but I thought I could mention it here as well)

[zmumi]

I re-considered adding something like AsyncCache I shared in the comments, but I still believe the memoization+wrapper pattern is the core of what this library tries to achieve

[MasterSergius]

Oh yeah, 4 years passed :) NVM, I'm glad you're fine.