Bug Report-AutoSklearn2 f1_macro Metric

[aoppongjnr]

Whilst fitting a model with AutoSklearn2Classifier, I run into this problem: `FileNotFoundError Traceback (most recent call last) in 3 from autosklearn.experimental.askl2 import AutoSklearn2Classifier 4 ----> 5 cls = autosklearn.experimental.askl2.AutoSklearn2Classifier(time_left_for_this_task=16200, 6 per_run_time_limit=None, 7 #resampling_strategy="cv-iterative-fit"n_jobs=-1,,

1 frames /usr/local/lib/python3.9/dist-packages/autosklearn/experimental/askl2.py in train_selectors(self, selected_metric) 354 self.this_directory / metric.name / "askl2_training_data.json" 355 ) --> 356 with open(training_data_file) as fh: 357 training_data = json.load(fh) 358 fh.seek(0)

FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.9/dist-packages/autosklearn/experimental/f1_macro/askl2_training_data.json'`
After navigating to '/usr/local/lib/python3.9/dist-packages/autosklearn/experimental, I noticed that f1_macro folder is missing. After doing a cat of askl2.py, I noticed that the metrics defined in this file does not have f1_macro. This is the definition of the metrics in the file: selector_metrics = (balance_accuracy,roc_auc, log_loss). Is it the case that I have to install a package in order to use autosklearn2? What I'm a missing on the metric? I'm running my code in Google Colab.

[aron-bram]

Hi, No, your setup is fine. However, the only metrics that will work with askl2 are (balanced_accuracy, roc_auc, log_loss). Unfortunately f1 doesn't have a related training_data.json file, as you also just pointed out.