Properly handle memout and timeout for quality optimization and memout for runtime optimization
PR #145 only fixes the bug that SMAC crashes if the first run is a timeout or a memout. If these occur more often, the location of these should be taken into account when suggesting new points. There are several solutions we could implement:
- https://arxiv.org/pdf/1403.5607.pdf
- https://arxiv.org/pdf/1004.4027v2.pdf
- https://pdfs.semanticscholar.org/2e07/9604c7a00c43f06e214280cea18a89dcecef.pdf
And a follow up issue which can only be solved once this is done: if we properly handle crashed runs, there is not always a reason to abort if the first run crashed. A crashing first run might occur with a random incumbent and a pipeline which has unknown constraints (for example an optimization algorithm in a ML method does not converge due to the dataset).
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
We will look into this together with #884.