confluent-kafka-python
confluent-kafka-python copied to clipboard
How do I get the the offset of last message of a Kafka topic
Description
I need to retrive last N messages of a topic using confluent-kafka-python.
I've been reading https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html# for a day, but no finding any appropriate method for getting the offset of the last message, thus I cannot calculate the offset for consumer to start with.
Please help. Thanks!
version: 1.8.2
libversion: 1.6.2
broker version: 3.0.0
client configuration: just {'bootstrap.servers': boot_strap_servers}
os: linux
Checklist
Please provide the following information:
- [x] confluent-kafka-python and librdkafka version (
confluent_kafka.version()andconfluent_kafka.libversion()): - [x] Apache Kafka broker version:
- [x] Client configuration:
{...} - [x] Operating system:
- [ ] Provide client logs (with
'debug': '..'as necessary) - [ ] Provide broker log excerpts
- [ ] Critical issue
Hi @tigerinus , thanks for asking. This is the method to retrieve low and high offsets for the specified partition: https://github.com/confluentinc/confluent-kafka-python/blob/master/src/confluent_kafka/src/Consumer.c#L1375