streamflow
streamflow copied to clipboard
Persistent topology monitor to ensure uptime of submitted topologies.
Currently, when a topology is submitted by a user from streamflow there are no protections that it will continue to run if there is a cluster outage.
This feature should implement a persistent service within the streamflow server that ensures that a topology that has been submitted will be resubmitted if the topology goes down for any reason. This will improve topology uptime in the event that a storm cluster goes down and comes back up for any reason.
Once a user triggers the kill operation in streamflow, the topology will be flagged as killed and should no longer be auto deployed.
The streamflow configuration should allow for enabling/disabling of this feature and controlling the polling interval of the service. A suggested config format is as follows:
monitor:
enabled: true
pollingInterval: 60