python-gatenlp
python-gatenlp copied to clipboard
Ray runner: better support for killing
Currently we just set a flag to abort the pipeline loop when Ctrl-C is pressed.
This will not work (well) with a pipeline which uses pipe() where a large number of documents are processed by the pipe.
One possible way: escalate the way we try to abort processing each time Ctrl-C is pressed and eventually forcibly kill all processes.
In addition, we could also trap the KILL signal on the main process and directly forward this to kill all processes.