Consuming messages from a simple consumer

[reutshal]

Hey, I'm trying to consume messages from a topic using pykafka with no duplicates. I'm interested in consuming messages every few minutes, so every time I will get only new messages.

I tried this (as a part of a function that will be called every few minutes):

but I'm getting the same messages every time.

I also tried to set the consumer like this:

with offset start from -2 and then update the offset according to the op.last_offset_consumed, but it still didn't work.

Basically I tried to perform a very trivial Kafka pattern- consume new messages.

Any ideas for this? I will appreciate your quick response.

Thank you!