storm-contrib
storm-contrib copied to clipboard
Fix out of range exception issue of partitioned Kafka spout
I modified KafkaUtils::emitPartitionBatchNew method to fix the out of range exception when fetch messages from Kafka's topic. This issue happens when there is no previous status in Zookeeper and the beginning offset of Kafka's brokers is non-zero.
-Binh
Any updates on this issue... Was this merged ?
I dont think this would fix anything. kafka.api.OffsetRequest.EarliestTime() = -2. If we set startOffsetTime = kafka.api.OffsetRequest.EarliestTime(); // which is the default behaviour. And set forceFromStart = true; // which is to be done in this case as well.
We could achieve same behaviour. Am I missing anything ?
Does this issue still exist in Storm 1.0.1 , as i am running with Storm 1.0.1 and got the same issue when i accidently deleted the kafka partitions . I had to create a new topic to avoid this issue and redeploy the topology with new topic as I was just doing POC.