heat icon indicating copy to clipboard operation
heat copied to clipboard

Published 20 hours ago • verified publisher icon helmholtz-analytics

Implement sparse matrices

Open Markus-Goetz opened this issue 5 years ago • 1 comments

Feature functionality Design and implement a distributed sparse DNDarray class. Come up with one or more reasonable storage formats (e.g. COO, CRS, CSC, ...)

Additional context PyTorch implements COO only at the moment with a very restricted operation set.

Interface design might be inspired by Armadillo and/or SciPy.sprarse

Markus-Goetz avatar Sep 13 '19 20:09 Markus-Goetz

Here is a great paper summarizing formats and operations on sparse tensors. I would go with a COO format first where the set of entries is roughly evenly distributed

http://groups.csail.mit.edu/commit/papers/2016/parker-thesis.pdf

Markus-Goetz avatar Apr 10 '20 10:04 Markus-Goetz

CSR addressed in #1028 , thanks @Mystic-Slice

ClaudiaComito avatar Nov 25 '22 10:11 ClaudiaComito

the ht.sparse module has been implemented by @Mystic-Slice and released with v1.3.0. For the time being it supports distributed CSR matrices and element-wise operations. More work needed.

Related issues: #1039 #1040 #1046 #1043 #1044

@AmirTouilPU has initiated work on distributed COO support, PR #988 is stuck for now.

I'll close this issue for now.

Reviewed within #1109

ClaudiaComito avatar Aug 04 '23 16:08 ClaudiaComito