Feature: add _HasApplyMixin and shared hyperparameters

[vincent-antaki]

Is your feature request related to a problem? Please describe. Sometime, we need multiple steps to share a hyperparameter. A simple example goes a little something like this : We have two ChooseOneStepOf at completely different places in the framework, both of which are basically a switch between different values of model_type (which is an hyperparameter we sample in an AutoML loop). We need the sample model_type value to be the same at all time.

To achieve this with the current state of the framework, we need to choose a step which will hold the hyperparam and have the other steps go retrieve the value at every call. Not only this conflicts with the concept that every step is responsible for keeping it own hyperparams / hyperparameters space but it is quite inelegant in term of code and not flexible at all in term of pipeline building.

Describe the solution you'd like Conceptually, I propose we add shared hyperparameters as an exception for the self-containing hp rule and store them in the context.

In more depth what I suggest is :

• Create a _HasApplyMixin which would be inherited from both BaseTransformer and ExecutionContext. This HasApplyMixin would have implement method get_name/set_name as well as defining an abstract apply interface.
• Add two arguments to the apply method : the execution context and a recursion depth argument.
• If depth is zero on a BaseTransformer apply call, then call apply on the context.

Upside :

• Shared hyperparameters are now easily accessible, at run-time, anywhere in the framework

Question left unanswered / Idea :

• How do we handle hyperparameter space / rvs for shared hyperparameter? We probably want context to implement _HasHyperparams and _HasHyperparametersSpace. I feel like this take us one step closer to the context being some kind of ultimate-all-encompassing-meta-step and I'm not sure I like it.
• How do we indicate to a step that it should fetch the required hyperparams in the shared hyperparam bank (i.e. the context)? I'd like to avoid re-coding a step to have a regular and a shared hyperparam version. (see alternative for a potential solution to this)

Describe alternatives you've considered We could have a special HyperparameterDistribution instance which simply go fetch the value to the responsible step (or the context if we decide to go with the solution mentioned above).

Additional context I feel like this would be a major modification, albeit a much needed one. I'd like your thoughs on this @guillaume-chevalier, I'm sure you can provide valuable insight on this design problem.

[guillaume-chevalier]

Exquisite.

How do we handle hyperparameter space / rvs for shared hyperparameter? We probably want context to implement _HasHyperparams and _HasHyperparametersSpace. I feel like this take us one step closer to the context being some kind of ultimate-all-encompassing-meta-step and I'm not sure I like it.

I just had an idea reading this: the context could hold some BaseTransformers (but not base steps). Thinking of the FeatureAggregation, I see that we would like some contextual steps in the services !

The context has a lot of feature envy from the transformers, and we already see it. Instead of coding new logic in the services dict, the context could be a TruncableStep with its own dict keys as names of contained steps, and dict values as the steps in question. One of this step could be a SharedHyperparameterBank. That wouldn't even be a BaseTransformer in fact... that would be a class with some hyperparams, a space, and apply methods.

This makes me feel that there is one more class between a BaseTransformer and its mixins that we could probably extract and use here.

Side note: a Truncable Step is too much for the context to inherit from. We want less maybe. That is perhaps a sign that the TruncableStep could be splitted furthermore into different mixins using the interface segregation principle to split the dict behavior appart (and the apply logic as well). That would be like an _IndexableStepMixin or something like that... perhaps?

How do we indicate to a step that it should fetch the required hyperparams in the shared hyperparam bank (i.e. the context)? I'd like to avoid re-coding a step to have a regular and a shared hyperparam version. (see alternative for a potential solution to this)

Yes, perhaps we could have a SharedHyperparamDistribution("name") class in the distributions and in the spaces classes that would contain only a string. Upon getting this class, the getter for the hyperparam and hyperparam space would know to retrieve it from the passed context's special hyperparam class. However, this would risk making the apply logic quite convoluted for the getters and setters. The exception we'd do at the level 0 of the applies for the context is already a lot for an exception. I would like to compare more solutions before picking the definitive one. I think we're on something.

[guillaume-chevalier]

@vincent-antaki

[vincent-antaki]

I'm not sure I see what's the point of having the context implement such TruncableStep-like interface. To allow reusage of a step at different point within the pipeline? If so, I see how it could be useful, but I think that may be overkill; we can probably find simpler way to do that through SharedHyperparamSample/SharedHyperparamDistribution.

I agree with the idea of a SharedHyperparameterBank. It would, in my opinion, be a good solution which keeps context relatively simple and creates a service with a very specific purpose.

As for the apply, I feel like all our options make it more complex. We could lose the level 0 exception by creating a static apply function in base.py which would take both pipeline + context + regular arguments, call apply on both context and pipeline (which would behave the same way it does now). As for the SharedHyperparameter... I don't see any way out with the current design.

Also, I'm wondering how much this change would affect issues #436 and #442

[vincent-antaki]

Somewhat linked: #316

Same issue: #211

[guillaume-chevalier]

Related: https://github.com/Neuraxio/Neuraxle/issues/516 For when the recursive context has global-level services used for shared hyperparams.

[guillaume-chevalier]

Related: #485 for apply on the context

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs in the next 180 days. Thank you for your contributions.