orion
orion copied to clipboard
Inconsistent algo.is_done based on max_trials
This test often fails with TPE because it converges faster than the other builtin algorithms and often end up suggesting 2 times the same trial (not exactly the same but due to precision default value [of 5 I think] they get rounded to same value). The bug is due to the fact that the inner algorithm will reach is_done
too early compared to what is requested from the user. The experiment often only has 29 trials whereas max_trials=29. This is inconsistent and confusing. The algorithm should compare number of trials to max_trials if it is wrapped.
I have a solution in mind:
Setting max_trials
on the SpaceTransform wrapper should only set it on the wrapper itself. It shouldn't be propagated down onto the wrapped algorithm.
We currently set that value with algo.unwrapped.max_trials
, it would instead be algo.max_trials = max_trials
, and the AlgoWrapper class would have a max_trial property with a custom setter that (by default) sets it on the wrapped algo. But in the case of that SpaceTransform wrapper, it would not get propagated down.