Question: is there an ask-tell interface?

[badrigopalan]

Description

Is there an ask-tell interface?

I would like to do step-by-step optimization manually by running previous optimization point and provide results to SMAC. Thank you!

[renesass]

Hey,

you can provide configurations to SMAC with the initial design (https://github.com/automl/SMAC3/blob/master/smac/initial_design/initial_design.py). However, an ask-tell interface is not implemented.

I'd happy if you can elaborate your idea further. :)

Best, René

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mb706]

I'm assuming @badrigopalan means the ask-tell-interface from skopt, what another optimizer calls "Human in the Loop" optimization?

So the idea would be to have the SMAC optimizer return the configuration(s) that should be evaluated next. The user can then do something with that configuration, e.g. run an expensive and error-prone simulation, or have human raters that evaluate the configuration. Finally, the SMAC optimizer function is called again with the result from that evaluation, returning the next configuration proposals.

I could imagine that having the target algorithm just be a print + abort, together with running SMAC with custom initial design could solve this to a degree. I.e. (1) SMAC calls algorithm (2) algorithm prints + aborts (3) human then evaluates the desired configurations and (4) human calls SMAC again with evaluated configurations as initial design. But I am wondering to what degree this behaves differently from what SMAC would do otherwise, e.g. does it still do intensification correctly?