ramp-workflow icon indicating copy to clipboard operation
ramp-workflow copied to clipboard

[RFC] Workflows refactoring

Open glemaitre opened this issue 5 years ago • 1 comments

The workflows should be derived from a metaclass to enforce having the parameter element_names.

I would like also to investigate:

  • If workflows should be derived from the scikit-learn Pipeline more generally.
  • The element of the workflow should have a tag _estimator_type to indicate which type they are. The integration with WorflowElementType would be eased.

glemaitre avatar Nov 27 '18 09:11 glemaitre

Good idea. As for deriving from Pipeline, why not. The simpler workflows could certainly be pipelined. I just don't think it could be done in general (e.g. drug spectra in which we have quite a bit of glue code between the steps. The flexible handling of CV folds is also something that makes it more general. Anyway, no reason not to investigate.

The WorflowElementType and the whole structure was introduced a bit prematurely. I thought we could syntactically support transfer learning by enforcing certain conventions. I'm less keen on this now, first there was no real demand for this, second, it's a bit dubious whether an element made for a certain problem can be reused in another. I would approach this from an AutoML point of view now. So your second bullet is a bit less important for me, I'd let the demand evolve a little bit.

kegl avatar Nov 27 '18 13:11 kegl