iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Kafka Connect: Commit coordination

Open bryanck opened this issue 1 year ago • 2 comments

This PR is the next stage in submitting the Kafka Connect Iceberg sink connector, and is a follow up to https://github.com/apache/iceberg/pull/8701, https://github.com/apache/iceberg/pull/9466, and https://github.com/apache/iceberg/pull/9641. It includes the commit coordinator and related tests.

Still not included for the sink are the integration tests, distribution build, or docs, which will be added in follow up PRs. For reference, the current sink implementation can be found at https://github.com/tabular-io/iceberg-kafka-connect, and you can read some existing docs at https://github.com/tabular-io/iceberg-kafka-connect/tree/main/docs.

bryanck avatar May 18 '24 14:05 bryanck

@ajantha-bhat @fqaiser94 Thanks for the reviews! I think I have addressed comments in case you want to take another look. @Fokko @nastra If either of you have time to take a look, that would be very appreciated.

bryanck avatar Jun 16 '24 16:06 bryanck

@rdblue, @danielcweeks, @Fokko, @nastra: Please take a look. It is needed for 1.6.0 milestone.

ajantha-bhat avatar Jun 28 '24 10:06 ajantha-bhat

I think I've addressed the latest feedback @fqaiser94 @danielcweeks LMK if there is anything else.

bryanck avatar Jul 11 '24 05:07 bryanck

Thanks @bryanck and @fqaiser94. It's really great to get this one in.

danielcweeks avatar Jul 11 '24 15:07 danielcweeks