spring-data-redis icon indicating copy to clipboard operation
spring-data-redis copied to clipboard

[Repository Pattern] Pin keyspaces to single slot only for index keys on cluster mode

Open sureshkmit opened this issue 11 months ago • 2 comments

Spring Data Redis repository pattern can confine only index keys within a single Redis slot when operating in cluster mode. Entity keys can be distributed across multiple slots as per the cluster's default hash algorithm. Commands (such as SINTER and SUNION) can still be processed on the server side as all involved index keys map to the same slot. Distributing entity keys across multiple Redis slots in a cluster environment can reduce hot spots and improved scalability and performance.

https://docs.spring.io/spring-data/redis/reference/redis/redis-repositories/cluster.html

Example NOTE: Curly brace only on index keys.

Key Type Slot Node
people:e2c7dcee-b8cd-4424-883e-736ce564363e id for hash 15171 127.0.0.1:7379
people:a9d4b3a0-50d3-4538-a2fc-f7fc2581ee56 id for hash 7373 127.0.0.1:7379
{people}:firstname:rand index 2399 127.0.0.1:7379
{people}:firstname:mike index 2399 127.0.0.1:7379

sureshkmit avatar Dec 17 '24 18:12 sureshkmit

If you would like us to spend some time helping you to diagnose the problem, please spend some time describing it. Right now, we struggle to understand what you want to achieve.

christophstrobl avatar Dec 18 '24 08:12 christophstrobl

Our current implementation utilizes the repository pattern to cache millions of objects, with indexing enabled on specific attributes. By confining all keys to a single Redis slot, we're effectively centralizing all data within a single shard. This approach negates the benefits of Redis Cluster mode, as horizontal scaling across multiple shards is hindered. To leverage the full potential of Redis Cluster, the library should distribute the objects data across the cluster. I believe restricting index keys to a single slot is adequate.

sureshkmit avatar Dec 18 '24 09:12 sureshkmit