[FEAT] Expose RINorm kwargs
Is your feature request related to a current problem? Please describe. This would largely address #2739 - RIN is not scale invariant
Describe proposed solution Exposing the RINorm hyperparameters would allow users more flexibility. Currently there can be unexpected behavior for samples with variance ~1E-5 -- low, but not so low as to be unrealistic. RINorm as reported will inevitably behave oddly with low variance, but epsilon is a key hyperparameter in this case and should be exposed.
Also, the affine transformation is unlikely to be beneficial with N-Linear and D-Linear models and we should allow users to disable it.
Finally, the default hyperparameters should be documented (they match the original implementation but may be unintuitive)
This solution could be made most simply by accepting a dictionary as an argument for the use_reversible_instance_norm kwargs in all PLForecastingModules, eg
if use_reversible_instance_norm and isinstance(use_reversible_instance_norm, dict):
self.rin = RINorm(input_dim=self.n_targets, **use_reversible_instance_norm)
elif use_reversible_instance_norm:
self.rin = RINorm(input_dim=self.n_targets)
else:
self.rin = None
Describe potential alternatives Nothing comes to mind, other than leaving things as-is.
Hi @eschibli, and thanks for this feature request. We could add this along with #2758.