CaImAn icon indicating copy to clipboard operation
CaImAn copied to clipboard

Enhancing dot product for sparse matrices during ROI spatial updating

Open oterocoronel opened this issue 1 year ago • 1 comments

When trying to refine the spatial components using the ellipse mode, these lines (lines 891-892 in spatial.py) made it crash because it couldn't allocate enough space in memory:

 cm[:, i] = old_div(
                    np.dot(Coor[c], A[:, :nr].todense()), A[:, :nr].sum(axis=0))

In particular, A[:, :nr].todense() doesn't seem to scale well with large recordings, since it requires allocating in memory a volume of shape = n_rois x dim1 x dim2 (x dim3). In my case, I am working with a 4D dataset and using CNMF-3D, so the size was ~ 10000 x 600 x 600 x 15 and that's why I ran OOM. But I think this may even be an issue for large/dense 2D recordings and/or users with limited RAM. I managed to overcome this issue by calling the "dot" function of the sparse matrix, so my new version looks like this:

 cm[:, i] = old_div(
                    A[:, :nr].T.dot(Coor[c].T).T, A[:, :nr].sum(axis=0))

Seems to be working just fine. Letting you guys know in case it's useful for others.

oterocoronel avatar Apr 12 '23 05:04 oterocoronel