milena
milena copied to clipboard
tylerholien
Support new-style groups
Now that I've gained an understanding of what is provided by the new style groups I am interested in using them. Do you have plans to complete the merge of your branch or would hours from another person be handy here?
Unfortunately I haven't been able to give this project the attention it deserves, so I would gratefully accept contributions!
Do you have any quick thoughts on tasks and completeness desired prior to merge?
The protocols are all implemented, so now it's a matter of implementing the API to use it. That being said, this could be merged right now, it just wouldn't be very useful.
If you are interested in implementing the group API, there is some documentation.
The most up-to-date info, but too low-level to be used by itself: https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.9+Consumer+Rewrite+Design
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61312126
However some of it is outdated and inconsistent, so I've been relying a lot on the Java and C client implementations:
https://github.com/apache/kafka
https://github.com/edenhill/librdkafka
I was thinking it would be a fun opportunity to learn about and experiment with session types, or at least play around with getting some of the protocol encoded in the type system, but I never got around to it. A port of one of the official clients is probably more prudent (though a lot less fun).