spatial_update with ellipse crashes for large 4D recording

[oterocoronel]

Your setup:

1. Operating System (Linux, MacOS, Windows): Linux
2. Hardware type (x86, ARM..) and RAM: 256 GB RAM
3. Caiman version (e.g. 1.9.12): 1.9.13
4. How you installed Caiman (pure conda, conda + compile, colab, ..): conda
5. Details:

I am trying to update the spatial components that I got after initializing with greedy_roi. My recording is 2660 frames of size ~700x600x15 (x,y,z) and I'm getting ~11,000 good ROIs. I want to use the 'ellipse' mode, since it has given me better results than the dilate mode. This works fine for ~9000 ROIs, but crashes systematically when having ~11000 ROIs. When crashing, Jupyter notebook and the terminal both close, so I don't get an error code. The last thing logged is:

170429 [spatial.py:update_spatial_components():185] [1049503] Computing support of spatial components

It seems like I should have enough space in memory to allocate more stuff: my recording is 70GB, my total RAM is 256GB, memory usage goes up to 60% before crashing:

I've used the 'dilate' method a few times with ~15000 ROIs and it seems to work just fine. I have noticed that for 'dilate' the memory usage is consistently lower (it never goes beyond 10-20%), while for the ellipse it slowly increases over time reaching 40%, and then increases even further.

My guess would be that at some point it is reshaping the results, and having to allocate an array of size nROIs x frame_size (700x600x15), possibly similar to #1060, but in this case it is hard for me to track down where is the actual root of the issue.

Any thoughts? Thanks!!