storm-jms icon indicating copy to clipboard operation
storm-jms copied to clipboard

duplicated emitted messages after a fail recover

Open frascuchon opened this issue 10 years ago • 1 comments

Hi,

Reading the documentation of the method session.recover(), I think the recovery process after a failure causes messages inside within the topology (after spout), are discarded by jms queue and re-send to spout. This causes duplicate messages to be processed.

Maybe an explicit emit after fail event can solve this problem (with some kind of retry policy)

I open this issue to discuss a possible solution.

Thanks in advance!

frascuchon avatar May 08 '14 14:05 frascuchon

As I see it, exactly once delivery guarantees are not to be expected. I certainly don't. Usualy Its a lot easier to have a idempotent block on your pipeline (eg. http://camel.apache.org/idempotent-consumer.html)

albertocsm avatar Aug 04 '15 09:08 albertocsm