Trust returned running time by target algorithm

[KEggensperger]

When using surrogate benchmarks for running time optimization SMAC3 does not used the returned running time by the benchmark to internally manage time budgets (e.g. intensification percentage).

We could: a) introduce an option b) always use max(measured runtime, returned runtime)

[mfeurer]

I think for this particular purpose it would be good to subclass ExecuteTARun in this file.

[mlindauer]

@mfeurer I don't think this solves the issue. Right now, we measure wallclock time in intensification to balance challenger selection and intensification. Since the wallclock time is much smaller when we use the katha's surrogates, the balance is broken. so, we have to change something in intensification.

[mfeurer]

Good point. If the goal is to move away from measured wallclock time we should do this in all places. Instead of measuring time in intensify, we could reuse the actual runtime that is stored in the runhistory. Then time would only be measured/faked in one place.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.