seb.

@RamlatchxRamspeicher we made several changes for `stable-v2` for the last paper, and it's very likely that the "old" report notebook is a bit out of sync now that we're using...

@pplonski no, this is not allowed to set parameters in this file, as we'll be using it for the AutoML Benchmark paper. We potentially accept only one high-level parameter like...

@pplonski the follow-up PR after my previous comment: https://github.com/openml/automlbenchmark/pull/193 I expect you want us to use `mljarsupervised_compete` in the next full benchmark run.

TODO: add description to HOWTO documentation.

@PGijsbers > I suppose automl packages could have their own metrics predefined to import, but if that is the case a scikit-learn fallback would not be enough either. I was...

> we might use e.g. hold-out for evaluation on large datasets instead of 5-fold CV sounds reasonable, we would probably also need to change the budget allocation in this case...

We started to support `string` dtype only quite recently in the openml loader, and the file loader hasn't been updated accordingly. But given the problematic raised by @yinweisu, we should...

> I disagree that the default behavior of the install should be to create a new venv in which to install rather than installing it in the active venv. I...

@PGijsbers yes, this is an old suggestion, and I agree with your pitfalls, so I would not consider this as a priority. Good to keep this for reference though.

The more I think about this issue, the less i'm convinced about it. First, to make this really useful, we need to implement in such a way that all jobs...