Sean Glover

@patilpratiks It sounds like you're describing an issue with the broker? Or are you observing messaging loss because of this exception on the client as described earlier in this issue...

> I am currently seeing the following dead letters message which is killing the stream but without any exceptions. The stream just dies silently. > > ``` > INFO 2020-02-26...

We discussed making a new experimental API for `RestartSource` to propagate materialized values better. I think that's worth experimenting within Alpakka Kafka. I don't think just wrapping the consumer sources...

There's likely an exception that's being swallowed. This is usually a result of the materialized value of the stream not being handled correctly. In OP's case they should handle the...

@ennru Right, I thought that might be a problem. Those docs are a big improvement! But I think it's still a little vague about the two suggestions I made in...

> it would be great if Alpakka would at least warn that offsets are commited in the wrong order and this may produce undesired / surprising results. Agreed, I created...

@silles79 When using `Producer.flexiFlow` we ensure ordering of `Envelope` (input) to `Result`, even when retries may occur (by using `mapAsync`). Therefore when used in combination with `Committer.sink` we shouldn't be...

@silles79 `Producer.flexiFlow` will only emit `Results` that have been successfully acknowledged by the broker (via a Kafka [Callback](https://kafka.apache.org/26/javadoc/org/apache/kafka/clients/producer/Callback.html)). Internally, the Alpakka Kafka Producer creates a `Promise` and its future is...

@PrathikPuthran have you tried using `enable.idempotence=true` with your `ProducerSettings` when using `Producer.committableSink`? I think this should resolve the issue you experienced. The setting has no impact on our internal benchmarks,...

@silles79 Your producer properties shouldn't be a problem. It's unclear from your code how the stream shuts down. Unless there's a cancellation signal from a stage, or an error occurs...