auto-sklearn icon indicating copy to clipboard operation
auto-sklearn copied to clipboard

Show a warning if too many evaluations fail

Open KEggensperger opened this issue 4 years ago • 2 comments

Describe the bug

If all evaluations crash, auto-sklearn does not even show a warning

To Reproduce

For example add a new model class which always crashes (or set n_jobs>1)

Expected behavior

If there are more failed than successful evaluations, I would appreciate a warning/critical warning/some output. Otherwise, a user might just use the Dummy estimator without knowing.

Actual behavior, stacktrace or logfile

Environment and installation:

KEggensperger avatar Nov 05 '21 13:11 KEggensperger

If all evaluations fail we could also print a few of the errors that we can parse from the runhistory to simplify debugging for the user?

mfeurer avatar Nov 10 '21 11:11 mfeurer

Closing related issue #1146 which says that a warning should be given if only the DummyModel is included in the final ensemble.

eddiebergman avatar Nov 17 '21 10:11 eddiebergman