DeepCAVE
DeepCAVE copied to clipboard
New plugin: Hardness of AutoML problem
Hi,
We often wonder how hard an AutoML problem is. Can we therefore add some metrics regarding that? For example
- an eCDF plot for the cost distribution (i.e., a hard AutoML task should have only a few very well-performing configurations)
- uni-modal metric from the automl loss landscape paper (but evaluated on our surrogate models)
- convexity metric from the automl loss landscape paper (but evaluated on our surrogate models)