poseidon icon indicating copy to clipboard operation
poseidon copied to clipboard

Is there no support for consumer groups?

Open messutied opened this issue 10 years ago • 2 comments
trafficstars

I'm sorry this isn't really an issue but I wasn't sure where to ask.

When creating a consumer instance the format is

consumer = Poseidon::PartitionConsumer.new(client_id, host, port, topic, partition)

Do I understand correctly that client_id is nothing related to a consumer group, and that poseidon does not currently support consumer groups?

If so, whats the purpose of the client_id? I see its used in the connection but I can't find references on the kafka documents to it.

Thanks

messutied avatar Jul 10 '15 21:07 messutied

+1 @bpot

ThomasAlxDmy avatar Jul 22 '15 03:07 ThomasAlxDmy

That is correct, an implementation of consumer groups is here: https://github.com/bsm/poseidon_cluster

It only covers some of the cases required for running a consumer, so make sure you test failure scenarios. A problem we have with it is that it does not handle NotLeaderForPartition errors and we have to restart consumers to fix it.

michaeldauria avatar Sep 16 '15 13:09 michaeldauria