workflow status update on scheduler crash

[oliver-sanders]

Sometimes workflows die in nasty ways (e.g. kill -9 <workflow-process> or poweroff). In these situtations the workflow does not shutdown cleanly, the process stops abruptly.

This means that the scheduler will not send a shutdown signal or any other message indicating that it is stopping.

We should test the UIS to ensure that when workflows crash the workflow state is correctly set to stopped (rather than waiting for the next scheduled scan to detect the issue). This may already be the case.