iceberg
iceberg copied to clipboard
Kafka Connect: Commit coordination
This PR is the next stage in submitting the Kafka Connect Iceberg sink connector, and is a follow up to https://github.com/apache/iceberg/pull/8701, https://github.com/apache/iceberg/pull/9466, and https://github.com/apache/iceberg/pull/9641. It includes the commit coordinator and related tests.
Still not included for the sink are the integration tests, distribution build, or docs, which will be added in follow up PRs. For reference, the current sink implementation can be found at https://github.com/tabular-io/iceberg-kafka-connect, and you can read some existing docs at https://github.com/tabular-io/iceberg-kafka-connect/tree/main/docs.
@ajantha-bhat @fqaiser94 Thanks for the reviews! I think I have addressed comments in case you want to take another look. @Fokko @nastra If either of you have time to take a look, that would be very appreciated.
@rdblue, @danielcweeks, @Fokko, @nastra: Please take a look. It is needed for 1.6.0 milestone.
I think I've addressed the latest feedback @fqaiser94 @danielcweeks LMK if there is anything else.
Thanks @bryanck and @fqaiser94. It's really great to get this one in.