[Caching] GetAllByPrefix like method

[etshei]

I'm trying to cache an ecommerce cart items using InMemoryCacheClient (dev) and in the future RedisCacheClient. My key is in the form S{sessionId}-V{productId} and the value contains a CartItem object (quantity and other related data) Adding and removing items are working great but i need to retrieve all the items for a sessionId. I know the existance of using the Set but i needed the lines separated. Is there a way to perform this? Maybe by using the ScopedCacheClient (i've done it with InMemoryCacheClient through the Keys prop but it will not work on RedisCacheClient)

[niemyjski]

I think you should be able to use a scoped cache client with the scope being the session id and then call GetAllAsync . I think this returns all items if no keys are passed in. Can you try this?

Couldn't you treat each set item as a line item or just store the whole cart object in cache and get it to update and set it again after incrementing the value?

[etshei]

I've already checked the code for GetAllAsync and it creates a dictionary with the keys provided as params (doesn't get all if not provided)

I've come to create a cart object (as you've suggested) and that worked (i had some issues to use the cache with Castle Windsor DI, that are now gone).

In another side, is there a way to provide a Sliding expiration mechanism in the future (re-setting the expiry time each time it's used)?

[niemyjski]

My apologies for the late reply, did you end up finding a solution for this?

[niemyjski]

@etshei were you able to find a solution for this?

[BlowaXD]

Hi, I'm having the same problem, I tried to have a HybridScopedCacheClient but it does not retrieve any of my objects.

Objects are stored with the right prefix (the one I set in my code)

Here is my constructor, I think i'm using it fine

Can I ask for some advices ? I was using ServiceStack.Redis but it needs to be paid for more than 6k requests per hour :(

[niemyjski]

Hello,

Can you please provide some more of your code with how you are calling to get the key starting with f575. If you do a getbyid does it work? Also are you connecting to the correct redis database instance?

We're doing over 16Million requests per hour with the Foundatio Redis cache client :). I'd first start out with just the RedisCacheClient (with Scoped) instead of the hybrid client (unless you have a lot of heavy objects where local serialization would be faster).

[BlowaXD]

Hi,

I totally forgot to share it yesterday. Of course, this is some examples I already tried : None of them worked :/

// first try (without arguments)
private async Task<IEnumerable<WorldServerDto>> GetServersAsync()
{
    return (await _cache.GetAllAsync<WorldServerDto>()).Where(s => s.Value.HasValue).Select(s => s.Value.Value);
}

// second try (with prefix + "*")
private async Task<IEnumerable<WorldServerDto>> GetServersAsync()
{
    return (await _cache.GetAllAsync<WorldServerDto>(Prefix + "*")).Where(s => s.Value.HasValue).Select(s => s.Value.Value);
}

I also took a look to the implementations, it looks like if you don't have any key provided, it won't return you anything. https://github.com/FoundatioFx/Foundatio.Redis/blob/master/src/Foundatio.Redis/Cache/RedisCacheClient.cs#L122 https://github.com/FoundatioFx/Foundatio/blob/master/src/Foundatio/Caching/ScopedCacheClient.cs#L70 https://github.com/StackExchange/StackExchange.Redis/blob/master/src/StackExchange.Redis/Interfaces/IDatabase.cs#L1806 https://github.com/StackExchange/StackExchange.Redis/blob/master/src/StackExchange.Redis/RedisDatabase.cs#L2248 https://github.com/StackExchange/StackExchange.Redis/blob/master/src/StackExchange.Redis/RedisDatabase.cs#L2254

[BlowaXD]

https://github.com/StackExchange/StackExchange.Redis/issues/1048#issuecomment-457373213 Following that suggestion I think it's a really clean way to do it (and does not require lot of code)

Should Foundatio provide something like that ? (It's already done in ServiceStack.Redis)

[niemyjski]

Yes you are correct. Currently we only have GetAllBy(keys) but no prefix. One would need to iterate all the keys on the server which could be a massive operation (and memory intensive) / would need to hit every server in the cluster). @ejsmith thoughts on adding this? Seems like if we did this you'd want to do some kind of paging but even then I don't even know how you'd accomplish that effiencetly.

[ejsmith]

It’s not really possible to do it efficiently. It requires a brute force approach and in a Redis cluster it requires iterating through the servers and asking them each for all their keys. I think the best approach is to maintain a set list of cache keys as you are adding / removing them. Then you can call that to get a list of all cache keys within a “cache group”.

[BlowaXD]

I personally made my own key cache (a simple set of existing keys) in my Foundatio wrapper but maybe this can be added inside the CacheClient to have a list of actual keys in the cache ?

[niemyjski]

I'd recommend doing what you are doing (which sucks), perhaps storing them in a set? It's not going to be very efficient to pull this off any other way. We have tens of thousands of keys in our cluster at any point. We are already looking into ways to do remove by prefix more efficiently / work better in a cluster environment. It would be nice to do this for say unit tests or small cache instances but how does this scale out to millions of keys without killing both Redis / cache implementation and your server when do you make this call?

As per your original question. Why not make the session be the set and then each line item be an entry in the set?

[ejsmith]

Yeah, I think it would be nice to be able to specify a cache group when you add or set a cache value and then be able to operations on them like delete by group.

[paulreicherdt]

A sliding expiration would be really nice. Even better - if you can set both, sliding and absolute expiration. Like within IMemoryCache