mbrl-lib
mbrl-lib copied to clipboard
facebookresearch
[Feature Request] Output Normalization / Scaling
🚀 Feature Request
When training non delta-state models, the outputs of dynamics models can take large values (way outside a unit Gaussian). In the past I have tried using output scalars to let the outputs try to learn something close to a unit Gaussian rather than variables with diverse scales.
Motivation
Is your feature request related to a problem? Please describe. I think it would help the PR for the trajectory-based model, #158 .
Pitch
Describe the solution you'd like I think there could be an optional output scalar that acts normally to the input one?
Are you willing to open a pull request? (See CONTRIBUTING) Sure.
Additional context
Add any other context or screenshots about the feature request here.
Not fully understand the normalization you have in mind. Are you referring to passing a set of constant scalars to be applied to the output of the dynamics model?
A set of scalars (can almost use the input normalizers) that map from the raw network outputs to the actual states of the environment.
Two times this was useful:
- Especially when using real-world data I found this could help with training convergence a lot (letting models stay in their proven region of mapping from things in the range of unit Gaussians to outputs of unit Gaussians).
- when using non delta-state models, as the model outputs can take really broad ranges of values.
Maybe its best for me to try it and see how it impacts some basic tests. Not a crucial addition.
Hi, is there any update regarding this? I have also used it in the past and found it to be useful in certain cases. Thanks!
@mohakbhardwaj -- I haven't made the time to make the PR. Happy to provide feedback if you take a stab at it?