spark-rabbitmq icon indicating copy to clipboard operation
spark-rabbitmq copied to clipboard

Published 20 hours ago • verified publisher icon Stratio

Data Loss on failure

Open kaveh-hariri opened this issue 7 years ago • 0 comments

Hi,

If the spark streaming job fails, the messages in the buffer will be lost. What are the current workarounds? Is it possible to only send an ack back for all the messages once the entire window is done processing (writing to a durable storage)?

kaveh-hariri avatar Oct 26 '17 22:10 kaveh-hariri