openml-labs
Use DASK to support caching and parallelism
Perhaps gama could switch from evaluating multiple pipelines single-core in parallel to evaluating one pipeline multi-core sequentially. This could lead to improved performance due to e.g. less memory use.
Besides the general pipelines, we currently train encoding/imputation models for every pipeline. While these steps are very quick, it is still wasteful. (this can also be prevented by instead doing preprocessing once in 5-fold CV and then transfer that dataset to search - although that does not generalize to multi-fidelity techniques without adaptations.)
Closing this because I don't think DASK is the right tool, and the issue is a bit too broad. Will open a new issue to look into pipeline caching.