storm-contrib icon indicating copy to clipboard operation
storm-contrib copied to clipboard

Fix out of range exception issue of partitioned Kafka spout

Open binhnv opened this issue 12 years ago • 3 comments

I modified KafkaUtils::emitPartitionBatchNew method to fix the out of range exception when fetch messages from Kafka's topic. This issue happens when there is no previous status in Zookeeper and the beginning offset of Kafka's brokers is non-zero.

-Binh

binhnv avatar Sep 28 '12 17:09 binhnv

Any updates on this issue... Was this merged ?

rsurendra avatar Dec 09 '15 14:12 rsurendra

I dont think this would fix anything. kafka.api.OffsetRequest.EarliestTime() = -2. If we set startOffsetTime = kafka.api.OffsetRequest.EarliestTime(); // which is the default behaviour. And set forceFromStart = true; // which is to be done in this case as well.

We could achieve same behaviour. Am I missing anything ?

kaustubhpriye avatar Mar 27 '16 08:03 kaustubhpriye

Does this issue still exist in Storm 1.0.1 , as i am running with Storm 1.0.1 and got the same issue when i accidently deleted the kafka partitions . I had to create a new topic to avoid this issue and redeploy the topology with new topic as I was just doing POC.

ersushantsood avatar Aug 03 '16 02:08 ersushantsood