auto-sklearn
auto-sklearn copied to clipboard
Show a warning if too many evaluations fail
Describe the bug
If all evaluations crash, auto-sklearn does not even show a warning
To Reproduce
For example add a new model class which always crashes (or set n_jobs>1)
Expected behavior
If there are more failed than successful evaluations, I would appreciate a warning/critical warning/some output. Otherwise, a user might just use the Dummy estimator without knowing.
Actual behavior, stacktrace or logfile
Environment and installation:
If all evaluations fail we could also print a few of the errors that we can parse from the runhistory to simplify debugging for the user?
Closing related issue #1146 which says that a warning should be given if only the DummyModel is included in the final ensemble.