Neuraxle
Neuraxle copied to clipboard
Feature: QueuedPipelines should be able to handle exceptions properly
Is your feature request related to a problem? Please describe. Workers should be able to catch exceptions properly and broadcast a termination signal to all of the current workers.
First thoughts about this ?
- Add a broadcast method to ObservableQueueMixin that sends a termination signal to all of the observers. https://github.com/Neuraxio/Neuraxle/blob/7cf9460eb2e971b32b4b84b8441d8515c9247d83/neuraxle/distributed/streaming.py#L90
- Broadcast exception the except block of the worker_function here: https://github.com/Neuraxio/Neuraxle/blob/7cf9460eb2e971b32b4b84b8441d8515c9247d83/neuraxle/distributed/streaming.py#L251
- Handle Continue exception in the QueueJoiner : https://github.com/Neuraxio/Neuraxle/blob/7cf9460eb2e971b32b4b84b8441d8515c9247d83/neuraxle/distributed/streaming.py#L713
- Handle the termination signal in the main thread to kill all processes and clean the workers....
Think about it that might not be the right solution...
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs in the next 180 days. Thank you for your contributions.