storm-hdfs icon indicating copy to clipboard operation
storm-hdfs copied to clipboard

Graceful Handling of Topology Kill

Open jimhe opened this issue 10 years ago • 0 comments

I'm wondering if there are any good approaches to ensure that tuples are flushed to HDFS when the topology is killed? Under normal operation, the bolt flushes according to the sync policy, but it acks every tuple even if they haven't been yet flushed. If the topology is shut down (storm issues kill -9 during a normal shut down) or if the bolt worker dies, the tuples left in its buffer are left untracked.

jimhe avatar Nov 20 '14 08:11 jimhe