mlaut

mlaut copied to clipboard

Integrate mlaut and sktime and ensure compatibility in the future * Port [strategies and tasks from sktime](https://github.com/alan-turing-institute/sktime/tree/master/sktime/highlevel) into mlaut. * Port current state of [benchmarking module from sktime](https://github.com/alan-turing-institute/sktime/tree/master/sktime/benchmarking) into mlaut....

ViktorKaz

Support different resampling strategies per dataset. The user can either provide a single resampling strategy that will be applied to all datasets or a list of resampling strategies each element...

ViktorKaz

triage/catwalk

1

the triage package now has a sub-package catwalk https://github.com/dssg/triage/tree/master/src/triage/component/catwalk (seems a few months old) it claims to do similar things as mlaut, and follows a similar but not identical design....

fkiraly

confidence intervals for all quantifiers

2

CI should be computed for average rank, and rank/performance differences as well. This should be easy to add as an incremental functionality.

fkiraly

re-factoring: estimator registry (incl off-shelf estimators)

1

currently, estimators are loaded via load_default_estimators that has an implicit in-code registry of available estimators and pipelines with default parameter settings and composition. It would be nicer if there were...

fkiraly

@frthjf has voiced concern over the defaults for DL not being sensible - the default choice for number of epochs for a keras network (1, i.e., one), which is the...

fkiraly

Would be nice to have as code becomes more complex.

fkiraly