CellSeg icon indicating copy to clipboard operation
CellSeg copied to clipboard

compute_stats() fails with large number of cells

Open qu4drupole opened this issue 2 years ago • 2 comments

I have processed CODEX data (that I assume came from the new PhenoCycler-Fusion system) saved as a single qptiff file. This format isn't quite what CellSeg expects for codex. With some small tweaks to the CellSeg code I was able to successfully complete the workflow on a cropped section (1000x1000) of the images. I used the 'CellSeg_stepbystep' notebook. The test image had an estimated 2.7k cells, and the overlays and statistics looked good. I reran everything on the entire image (~200k cells) and my kernel crashes at walkthrough_utils.compute_stats. The Jupyter log isn't very helpful:

[I 2022-05-20 14:44:19.756 ServerApp] Saving file at /CellSeg_stepbystep.ipynb [W 2022-05-20 14:44:19.757 ServerApp] Notebook CellSeg_stepbystep.ipynb is not trusted [I 2022-05-20 15:05:30.095 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (1/5), keep random ports kernel 815a70eb-3373-413b-84f1-3fa4c437534a restarted kernel 815a70eb-3373-413b-84f1-3fa4c437534a restarted [I 2022-05-20 15:05:30.168 ServerApp] Starting buffering for 815a70eb-3373-413b-84f1-3fa4c437534a:b07fe2b1-dfed-4a1f-a482-1c7f08a65774

I tried using more memory (500G). No avail. Any idea on what's causing the problem? What information would be helpful for troubleshooting?

qu4drupole avatar May 20 '22 21:05 qu4drupole