Use Redis multiple get operation

[ceremo]

Redis repositories are based on KeyValueRepository, this interface defines Iterable<T> findAllById(Iterable<ID> ids), and the implementation defines a loop executing a findById per id (SimpleKeyValueRepository), but Redis could instead use the operation MGET to execute in a single request to the Redis server all requested ids.

Have you analyzed this alternative? Or maybe adding an extra method like Iterable<T> multiFindById(Iterable<ID> ids) taking advantage of the operation MGET?

Thanks for your time.

[dfidalgo-jpm]

Hello, I'm interested in this. I am using spring-data-redis repositories and findAllById is very slow. As I cannot use MGET for that, I switched to something like:

redisTemplate.executePipelined { connection: RedisConnection ->
                val serializer = redisTemplate.stringSerializer

                for (key in keys) {
                    serializer.serialize(key)?.let { connection.hashCommands().hGetAll(it) }
                }
                null
            } 

But it would be nice if it was baked in the provided repositories.

[rheeri]

Hi @mp911de , I’m interested in contributing to this issue. Do you have any initial ideas or a preferred approach for this feature? If not, I'm happy to explore solutions and put together a draft PR!

[juliojgd]

@rheeri @mp911de Be aware that in order to use the MGET Redis operation (line in redisTemplate.opsForValue().multiGet(lKeys)) you have to have ALL the keys requested in the same slot. If you want to have keys in the colletion in different slots you should use other approach like pipelining (like redisTemplate.executePipelined(SessionCallback))

The pipelining approach does not offer all the performance improvement (it improves over a loop looking for individual keys) of using MGET but provides a wider applicable use case scenario, as MGET imposes that all entries with the keys in the list/collection MUST be in the same slot. So it's more general the "pipelining" approach.

[tishun]

The underlying driver might handle the crosslot aggregation. Lettuce does, Jedis seems to lack this functionality.

[mp911de]

We do provide crossslot-agreggation in Spring Data Redis for Jedis MGET. This change has some pre-requisites in Spring Data KeyValue, see spring-projects/spring-data-keyvalue#655.