Viktor Yanush

RMSprop is used in RNNs quite often, we should support it too.

We can make it as in [weight_norm](https://pytorch.org/docs/stable/_modules/torch/nn/utils/weight_norm.html): delete old `nn.Parameter`, re-register new `ManifoldParameter`. Also, we need forward pre-hook because, for example, `Stiefel` is defined only for 2d-tensors. So, probably we...

> Do I correctly understand that this setting is indeed allows to treat manifolds independently? Yes.