[QUESTION] How to avoid missing messages between calls to AIOKafkaConsumer.subscribe

[andrew222651]

trafficstars

If we're subscribed to channels "a" and "b" and we want to drop "a" and keep "b", the docs for AIOKafkaConsumer.subscribe say "Topic subscriptions are not incremental: this list will replace the current assignment (if there is one)" so we would just call subscribe to subscribe to "b" only. But is there the possibility that we'll miss a message to "b" in between being subscribed to "a" and "b" and being subscribed to "b"? Is there a way to avoid this?

[ods]

Do you commit offsets manually? If so, the new subscription starts from the last committed offset for each topic, so nothing is lost even if you resubscribe after delay.

[andrew222651]

Ok I think I was a little confused. When subscribe is called, the consumer starts from its last committed offset on each topic, right? If I'm using automatic committing, can I still call consumer.commit manually?

[ods]

Automatic committing in aiokafka itself can cause message loss. See #826 for more info.