distilabel
distilabel copied to clipboard
Create `CTRL + C` signal handler for gracefully pipeline stopping
Create a signal handler that captures SIGINT
(ctrl + c) and stops the pipeline gracefully.
This consistently stops me from being able to stop the execution of a pipeline if or when it got stuck somewhere. Would probably be best to have some override like exiting if crtl+c was pressed more than once or twice.
To reproduce this @plaguss you can just create an infinite running step as:
import time
from distilabel.steps import StepInput, step
from distilabel.steps.typing import StepOutput
@step()
def InfiniteStep(*inputs: StepInput) -> StepOutput:
time.sleep(100)
And the CTRL+C will be captured, but the SIGTERM signal won't be sent.
Hi @bjoernpl in the next release this should be tackled: https://github.com/argilla-io/distilabel/pull/630