Kafka messages were processed twice after rebalancing
Checklist
- [-] I have included information about relevant versions
- [-] I have verified that the issue persists when using the
masterbranch of Faust.
Steps to reproduce
Tell us what you did to cause something to happen.
I have 1 replica of a faust streaming service that listens to a kafka topic with 4 partitions. This service is currently processing kafka messages when our autoscaler scales it into 2 replicas which causes the rebalancing. Before rebalancing, the first replica processed 3 messages, and after rebalancing, the same 3 kafka messages were reprocessed in the second replica.
Expected behavior
I expect that the 3 messages that were already processed in the first replica should not be reprocessed in the second replica since the these messages should be committed before rebalancing
Actual behavior
These 3 messages were processed twice. The first time in the first replica and the second time in the second replica.
Versions
Python version: 3.7.13 Faust version: 0.10.12 Operating system: Debian GNU/Linux 10 Kafka version: 2.3.1