Stopping condition for cost sensitive evaluations

[mmahsereci]

Currently the stopping condition is only checked at beginning of each loop-step. If the evaluation is cost/budget sensitive e.g., in multi-fidelity/source models or models where the input location determines the evaluation cost, then we might want to check after point calculation, too. Alternatively the point calculator could be smarter and only return points which are still below budget, but that is harder to do.

[marpulli]

Do you have ideas how to do this? We should at least think how we might do it before 1.0 is released in case it requires changes to the interfaces of any classes.

[javiergonzalezh]

I think that there are two different things here. One is the stoping condition itself, which is well defined, and the other one is how the decisions about the next points to collect are affected by that.

If we think about a simple case of making non-myopic decisions just using the remaining evaluation budget (no cost) what we need to do is to pass the state of the loop into the acquisition and handle the logic of the decision inside.

The case of having costs is harder because the stopping condition needs to know if given the current state (knowledge about the cost) the acquisition is going make a decision that is feasible within the given budget (otherwise we'll incur into a cost violation). Following the discussion in issue #25 this could be handled in the customizable part of the candidate_point_calculator and include the stoping logic there.