CaImAn
CaImAn copied to clipboard
Enhancing dot product for sparse matrices during ROI spatial updating
When trying to refine the spatial components using the ellipse mode, these lines (lines 891-892 in spatial.py) made it crash because it couldn't allocate enough space in memory:
cm[:, i] = old_div(
np.dot(Coor[c], A[:, :nr].todense()), A[:, :nr].sum(axis=0))
In particular, A[:, :nr].todense()
doesn't seem to scale well with large recordings, since it requires allocating in memory a volume of shape = n_rois x dim1 x dim2 (x dim3). In my case, I am working with a 4D dataset and using CNMF-3D, so the size was ~ 10000 x 600 x 600 x 15 and that's why I ran OOM. But I think this may even be an issue for large/dense 2D recordings and/or users with limited RAM. I managed to overcome this issue by calling the "dot" function of the sparse matrix, so my new version looks like this:
cm[:, i] = old_div(
A[:, :nr].T.dot(Coor[c].T).T, A[:, :nr].sum(axis=0))
Seems to be working just fine. Letting you guys know in case it's useful for others.