Lars Kotthoff

Please provide a reproducible example. It looks like there's a typo, the measure, learner, and task are not compatible, and you haven't defined the tuner.

Thanks -- internally, mlr3 uses `future_mapply()`, which, like the base R function `mapply()`, doesn't guarantee any particular order of execution. What's the use case for needing a particular order?

Also @MasonGallo if you feel like it.

Thanks, any help welcome.

You can save the best classifier by saving the model when Auto-WEKA completed its run, just like for any other WEKA classifier. Is this what you're looking for?

Hmm, you might be able to get the best model from the timestamp of the file, but Auto-WEKA doesn't explicitly keep track of this unfortunately.

Thanks. I'm getting an exception when no configurations were evaluated: ``` java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 java.util.ArrayList.rangeCheck(ArrayList.java:653) java.util.ArrayList.get(ArrayList.java:429) autoweka.ConfigurationCollection.forceFirst(ConfigurationCollection.java:175) autoweka.ConfigurationCollection.rank(ConfigurationCollection.java:139) weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:377) weka.gui.explorer.AutoWEKAPanel$11.run(AutoWEKAPanel.java:411) at java.util.ArrayList.rangeCheck(ArrayList.java:653) at java.util.ArrayList.get(ArrayList.java:429) at autoweka.ConfigurationCollection.forceFirst(ConfigurationCollection.java:175) at autoweka.ConfigurationCollection.rank(ConfigurationCollection.java:139)...

I think I tried this with the yeast dataset from the Auto-WEKA paper. Presumably you tested with all of them to measure the performance improvement with ensembles?

Great, thanks! Do you have the performance comparison with and without ensembles as well?

No worries. I've run some small experiments myself and didn't see any improvement with ensembles, so I'm curious to see your experiments.