kafka adapter:

[robambalu]

• Major chagne is to allow multiple consumers / consumer threads to service the same topic
• Each consumer can now get assigned a subset of partitions for a topic. Consumers need to keep track of their own partitions
• Since we dont know which keys are on which partition, the subscriber map is moved from Consumer to AdapterManager which will have the full set of subscribers for a given topic
• Removed complexity of not subscribing to Rebalance events if not using start offset, always sub and wait for EOF even if not using start offset
• exposed kafka consumer and producer specific config options

[ptomecek]

Should we get https://github.com/Point72/csp/pull/8 over the line so we can include the Kafka tests as part of CI? I think it was mostly ready to go except for the one test with a race condition which had just been disabled.

[robambalu]

Should we get #8 over the line so we can include the Kafka tests as part of CI? I think it was mostly ready to go except for the one test with a race condition which had just been disabled.

Yes absolutely, I didnt even realize the kafka test was completely disabled. Its heavily relied upon and should be tested