Emit messages from TransactionalFlow only after committing them

[szymonm]

Purpose

Resolves #791

Changes

Introduces waiting for commit message before emitting from an element from the TransactionalProducerStage.

This has the following effect on the logic: After the TransactionalProducerStage we have mapAsync(txSettings.parallelism). Before in the mapAsync stage, we used to wait for Kafka ack message (producer.send callback). Now we are going to wait for the transaction to complete (producer.commitTransaction). Thus we need to increase the txSettings.parallelism to accommodate committing many messages in the same transaction.

What do you think about this change @2m, @ennru ?

[2m]

Interesting approach! It would be useful to play with the parallelism and see how it affects performance, as the smaller it is the quicker mapAsync buffer will fill in and the stream will backpressure until transaction is committed.

Transaction benchmarks will be usable for this after https://github.com/akka/alpakka-kafka/pull/804 gets in.

[patriknw]

@szymonm Seems like this has been stale for a long while. Shall we close as obsolete?

[szymonm]

I think that #791 is valid and this is the right solution.

I can resolve the conflicts and improve the code as @2m suggested, if you agree with me in principle.

[ennru]

We are close to merging the use of KIP-447 transactions which solves this.

• #1728