kafka-python icon indicating copy to clipboard operation
kafka-python copied to clipboard

Add a default Protocol to the JoinGroup Structs

Open jeffwidman opened this issue 8 years ago • 1 comments

In the JoinGroup structs, the consumer can pass a protocol that it supports. However, if you read the explanation behind this design, it says:

The primary use case for the membership API is consumer groups, but the requests are intentionally generic to support other cases (e.g. Kafka Connect groups). The cost of this generality is that specific group semantics are pushed into the client. For example, the JoinGroup/SyncGroup requests defined below have no explicit fields supporting partition assignment for consumer groups. Instead, they contain generic byte arrays in which assignments can be embedded by the consumer client implementation.

But although this allows each client implementation to define its own embedded schema, compatibility with Kafka tooling requires clients to use the standard embedded schema used by the client shipped with Kafka. The consumer-groups.sh utility, for example, assumes this format to display partition assignments. We therefore recommend that clients should follow the same schema so that these tools will work for all client implementations.

Since the assumption is that all clients implement the same protocol as Kafka's Java client, we should add this Protocol as a class and then set it as a class var in the JoinGroup structs... this would make it easier to use the JoinGroup structs with the low-level KafkaClient because you can easily specify the default protocol rather than having to lookup how to do this...

jeffwidman avatar Feb 03 '17 20:02 jeffwidman

The structs for this are currently in kafka.coordinator.protocol . We could move these to kafka.protocol.group or just link from there to where they are now (as long as there is no circular import issue).

dpkp avatar Mar 07 '17 22:03 dpkp