protoactor-dotnet icon indicating copy to clipboard operation
protoactor-dotnet copied to clipboard

Introduce infra and examples on sharding

Open rogeralsing opened this issue 2 years ago • 2 comments

Akka Cluster Sharding is somewhat similar to virtual actors. instead of virtual actors, you have shards, which then, in turn, contain entities.

The cons of sharding are that it's not very fine-grained, you can get unbalanced workloads across nodes. The pros are that it is more lightweight on topology changes.

e.g. instead of moving millions of virtual actors around. you move far fewer shards. (in total there would be as many entities as virtual actors)

This would not load the PID Cache and identity lookup as much as virtual actors.

To build this, you would basically have a Shard virtual actor. that then routes messages to the correct child entity actor.

It could also take advantage of batching for message delivery. e.g. messages to entities in the same region could be grouped together and sent together, and potentially Acked back together. Where the shard region virtual actor could then use fire and forget messaging to the child entities.

image

rogeralsing avatar Feb 26 '22 12:02 rogeralsing

Like so Skärmavbild 2022-02-26 kl  14 00 51

rogeralsing avatar Feb 26 '22 13:02 rogeralsing

I found this today: Akka can do Local Affinity processing of Kafka partitions: https://www.youtube.com/watch?v=Ad2DyOn4dlY

We should add info about this in the local affinity docs and describe the similarities/differences

rogeralsing avatar Mar 02 '22 04:03 rogeralsing