GPU support

[mateuszbaran]

I checked how hard would it be to add GPU support to Manifolds.jl. Essentially without some kind of contextual dispatch (like this: https://github.com/JuliaGPU/CUDAnative.jl/pull/334 ) it would require rewriting almost everything.

[kellertuer]

I would have to first read about GPU support in general, but yes it would be great, especially for large power manifolds to have GPU support.

[mateuszbaran]

I'm not even sure what would I recommend reading here, usually GPU support is done from the other side than I'd prefer to do it here: each manifold is implemented to also handle its own powers (e.g. Pymanopt, Geomstats). That puts on us the burden of handling power manifolds separately for each manifold, doesn't scale to product manifolds and is troublesome when someone cares about performance in low-dimensional cases.

Here I'd prefer to just rewrite PowerManifold to automatically generate GPU kernels from CPU implementations of methods. That's not always possible but with contextual dispatch we should be able to automatically offload to GPU most non-allocating operations (GPU code can't really allocate).

[kellertuer]

Thanks for the explanation, I actually still prefer our generic approach to power manifolds and rewrite that one, too.

[mateuszbaran]

Related thing: https://github.com/JuliaGPU/KernelAbstractions.jl .

[sethaxen]

Yeah KernelAbstractions that was just recommended to me and seems like it might be promising.