alpakka-kafka icon indicating copy to clipboard operation
alpakka-kafka copied to clipboard

Possiblity to inject a Strategy to KafkaClusterSharding

Open mehmetsalgar opened this issue 2 years ago • 0 comments

Short description

Kafka Cluster Sharding is really useful to reduce the amount inter shard messaging but at the moment it is only doing a modulo operation over the entityId, it would be nice to be able insert a strategy/logic to compute the shardId.

Details

What I try to achive, I have a following constellations with my Actors.

ActorA_instance1 <-> id: 123456789 ActorB_instance1 <-> id: 123456789_XXXXXXXX ActorB_instance2 <-> id: 123456789_YYYYYYYY ActorC_instance1 <-> id: 123456789_ZZZZZZZ

And these Actor are collobrating over an Use Case of mine and there would be message exchanges between those but with current sharding logic they would be %99 land on different shards.

What I want is to have the control to be able to say KafkaClusterSharding, all Actors that have '123456789' land in the same shard, so possiblity to pass a function to calculate shardId.

At the moment, while KafkaClusterSharding is final, I have to duplicate lots of code to achieve my goal.

It would be really nice to have a possiblity to pass a function to calculate the shardId.

mehmetsalgar avatar Apr 28 '22 13:04 mehmetsalgar