MLOS
MLOS copied to clipboard
SMAC random forest model seems non-deterministic
trafficstars
Changes from #470 made max_trials be respected, so that n_random_init is by default very small for some tests.
This exposed some non-determinism in the pyrfr module that SMAC uses by default for its RandomForest model.
Essentially, on occasion, the value the model suggested would differ (repeatedly) across versions of Python or platforms (Linux vs Windows).
This was rather hard to track down, so for now we simply altered the tests to only use (seeded) random configs.
We should probably track that down at some point though so our model based optimizer suggestions are reproducible given the same inputs.