Functionality that enables autoscaling

[eelkevanfoeken]

Not sure if a suggestion belongs here. But for us it would be nice when it would be possible to have autoscaling of the number of input peers based on the number of instances.

For example: There is a kafka cluster with a topic with 100 partitions. The number of peers that read from kafka increases linearly up to 100 with increasing numbers of instances.

Or at least that you do not have to set the n-peers or min/max-peers, but that the partition-assigment function does the partitioning based on the eventual number of peers.

Do you think these functionalities lie in the realm of possibilities?