python-gatenlp icon indicating copy to clipboard operation
python-gatenlp copied to clipboard

Ray runner: better support for killing

Open johann-petrak opened this issue 3 years ago • 0 comments

Currently we just set a flag to abort the pipeline loop when Ctrl-C is pressed.

This will not work (well) with a pipeline which uses pipe() where a large number of documents are processed by the pipe.

One possible way: escalate the way we try to abort processing each time Ctrl-C is pressed and eventually forcibly kill all processes.

In addition, we could also trap the KILL signal on the main process and directly forward this to kill all processes.

johann-petrak avatar Mar 29 '22 08:03 johann-petrak