orion icon indicating copy to clipboard operation
orion copied to clipboard

Inconsistent algo.is_done based on max_trials

Open bouthilx opened this issue 2 years ago • 2 comments

This test often fails with TPE because it converges faster than the other builtin algorithms and often end up suggesting 2 times the same trial (not exactly the same but due to precision default value [of 5 I think] they get rounded to same value). The bug is due to the fact that the inner algorithm will reach is_done too early compared to what is requested from the user. The experiment often only has 29 trials whereas max_trials=29. This is inconsistent and confusing. The algorithm should compare number of trials to max_trials if it is wrapped.

bouthilx avatar Jul 05 '22 19:07 bouthilx

I have a solution in mind: Setting max_trials on the SpaceTransform wrapper should only set it on the wrapper itself. It shouldn't be propagated down onto the wrapped algorithm.

lebrice avatar Jul 15 '22 21:07 lebrice

We currently set that value with algo.unwrapped.max_trials, it would instead be algo.max_trials = max_trials, and the AlgoWrapper class would have a max_trial property with a custom setter that (by default) sets it on the wrapped algo. But in the case of that SpaceTransform wrapper, it would not get propagated down.

lebrice avatar Jul 15 '22 21:07 lebrice