Matthias Feurer

Add model export to sklearn

1

Fixes #388 by adding a function `to_sklearn`. This is a first draft for the interface, and I'd be very happy about feedback. # TODOs * [ ] unit tests *...

Text preprocessing V2 TODOs

4

This is a list of follow-up tasks to #1300. # General implementation * [x] Improve text example to include more meaningful dataset * [x] Improve text example to contain links...

This is mostly a discussion item. PR #1402 unveiled a bug in the handling of pandas that could lead to drastically worse performance (see [here](https://github.com/automl/auto-sklearn/runs/5200936217?check_suite_focus=true)). It was uncovered because the...

Provide multi-class AUC ROC

7

Scikit-learn provides multi-class options for area under curve: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html We should provide the most common ones, such as the [OVO Macro averaging used by Auto-Gluon](https://github.com/awslabs/autogluon/blob/0b38dde5f698dbadfa1ce76aabda14505d9e3ead/core/src/autogluon/core/metrics/__init__.py#L443).

Avoid costly re-building of pipelines

2

Currently, SMAC suggests hyperparameter configurations which are independent of the dataset size. For example, the hyperparameter `classifier:max_features` which is specified between zero and one is transformed according to `max_features =...

Scikit-learn's model selection modules such as [RandomizedSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html#sklearn.model_selection.RandomizedSearchCV) and [HalvingRandomSearchCV](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.HalvingRandomSearchCV.html#sklearn.model_selection.HalvingRandomSearchCV) use the feature name `scoring` instead of `metric`. We should follow scikit-learn here and deprecate the argument `metric` and replace it...

As part of statistics on the search procedure.

The current API, the [Auto-sklearn 2.0 paper](https://arxiv.org/abs/2007.04074) nomenclature and scikit-learn nomencature are different: | current API | Auto-sklearn 2.0 terminology | sklearn | should be named | |---|---|---|---| | `resampling_strategy`...