MLOS icon indicating copy to clipboard operation
MLOS copied to clipboard

SMAC random forest model seems non-deterministic

Open bpkroth opened this issue 2 years ago • 0 comments
trafficstars

Changes from #470 made max_trials be respected, so that n_random_init is by default very small for some tests. This exposed some non-determinism in the pyrfr module that SMAC uses by default for its RandomForest model. Essentially, on occasion, the value the model suggested would differ (repeatedly) across versions of Python or platforms (Linux vs Windows). This was rather hard to track down, so for now we simply altered the tests to only use (seeded) random configs. We should probably track that down at some point though so our model based optimizer suggestions are reproducible given the same inputs.

bpkroth avatar Aug 10 '23 22:08 bpkroth