Akka.Streams.Kafka icon indicating copy to clipboard operation
Akka.Streams.Kafka copied to clipboard

Fix TransactionalProducer implementation once Confluent driver support this

Open IgorFedchenko opened this issue 6 years ago • 5 comments

There are couple of TODO labels in TransactionalProducerStage.cs file (in TransactionalProducerStageLogic class implementation). This is due to lack of transactional operations support in Confluent driver's producer implementation.

This also requires some librdkafka update. The issue to track is https://github.com/confluentinc/confluent-kafka-dotnet/issues/1091.

Once this will be ready, implementation should work (there is a test at TransactionalIntegrationTests.cs that is skipped right now).

IgorFedchenko avatar Oct 30 '19 15:10 IgorFedchenko

Looks like they have published update which includes transaction support: https://www.nuget.org/packages/Confluent.Kafka/1.4.0

So now we can implement missing transactional stages.

IgorFedchenko avatar Apr 25 '20 11:04 IgorFedchenko

ah, this is a long waiting feature that I'm waiting to be implemented

chris-sung avatar Apr 10 '21 01:04 chris-sung

@chris-sung Yeah, the problem is that _producer.InitTransactions call to Confluent driver is blocking for some reason, and I can't make my PR passing tests. And I ran out of time, not able to fix it now... If you will have time and willing to check it out or fork my branch and play with this, you are wellcome :)

Otherwise it may take a long time to have this working...

IgorFedchenko avatar Apr 10 '21 19:04 IgorFedchenko

Hi, Is this resolved now? or still pending?

rafalpiotrowski avatar Jul 19 '22 10:07 rafalpiotrowski

Hey, nope, this is not resolved - there is a PR (https://github.com/akkadotnet/Akka.Streams.Kafka/pull/196) which was never finished (see comments above). Probably it will work now with latest driver versions, not sure.

IgorFedchenko avatar Jul 19 '22 10:07 IgorFedchenko