poseidon
poseidon copied to clipboard
Sporadic NoMethodErrors when calling Poseidon::PartitionConsumer.consumer_for_partition
I'm using Poseidon::PartitionConsumer.consumer_for_partition to get a PartitionConsumer on a cluster of kafka servers. Every once in a while it will blow up with a stack trace like this:
NoMethodError: undefined method `partition_leader' for nil:NilClass
from poseidon/partition_consumer.rb:26:in `consumer_for_partition'
from poseidon/broker_pool.rb:12:in `open'
from poseidon/partition_consumer.rb:30:in `block in consumer_for_partition'
from poseidon/cluster_metadata.rb:52:in `lead_broker_for_partition'
My calling code is like so:
def consumer
@consumer ||= Poseidon::PartitionConsumer.consumer_for_partition(@app, kafka_servers,
@topic, @partition, @offset)
end
def kafka_servers
["host1:9092", "host2:9092", "host3:9092"]
end
I'm able to rescue and retry (at which point it recovers), but it seems like a bad thing. What am I doing wrong?