Support for Sphere/SO(3)/SE(3)

[richardrl]

Hi, is there support for these manifolds?

[richardrl]

Also, is there anything manopt can do that mctorch cannot do? I've been using pymanopt to optimize over Sphere(4) but it's very slow (CPU) and doesn't easily support batched computations, so I am wondering if mctorch can be an almost-drop-in replacement for pymanopt

[bamdevm]

Hello Richard,

Thanks for the interest in McTorch.

We can handle Sphere and SO (assuming you are looking for the orthogonal group). The manifold to use is Stiefel. Say St(n,p) stands n-by-p matrices. Then, for sphere p = 1 and orthogonal group p = n.

We currently do not have support for SE3.

In general, it should be quite easy to port manifold definitions from Pymanopt to McTorch.

The motivations for Manopt and McTorch are different. McTorch is mostly for deep learning applications, whereas Manopt or Pymanopt has wider applicability.

Regards,

[richardrl]

Wonderful! How hard would it be to implement the product manifold?

Also, in my case, I am learning a neural network cost function C(s,"a") where "a" is my decision variable, and then optimizing over a manifold for "a". In my situation McTorch would be better suited because of its GPU support right?

[bamdevm]

By a product manifold, if you mean, say, two weight matrices on different manifolds, then it is directly possible. For example, if the weight matrices are W1, W2, W3, ..., then McTorch allows W1 \in M1, W2 \in M2, W3 \in M3, .... So, the product structure flows through naturally.

For your other question, McTorch should be better as it exploits GPUs.

Let us know your experience.

[richardrl]

Hi Bamdev, Great to hear. What's the best way to initialize 10 vectors where each one is in Sphere(4) manifold?

[bamdevm]

stiefel(4,1,10).