[BUG] Gaps in interphase and mitotic 3D nuclei when gradXY flows and cell probability look fine

[derekthirstrup]

I have found in my testing that the main branch Cellpose 4.0.6 has a bug where 3-D mask z slices can “disappear” in single or range of z-planes even when cellprob is high (red) and flows look reasonable. This happens with both large interphase nuclei and more often mitotic nuclei. I implemented a dynamic 3d bounding box cropping/ds processing script that speeds up 3D volumetric segmentation jobs 10-20 fold using new cellpose sam model. I added tracking of 3D nuclei using bipartite graphs over time but this segmentation issue is causing track linking errors since mitotic objects get split by this bug and interphase nuclei sometimes are impacted.

Here are the most likely sources of this issue but need more work to determine a robust implementation solution since this fine tuned model works well with exact same slices in 2D mode indicating it is not the model.

1. 3-view fusion is an unweighted sum In core.run_3D, the flows from YX, ZY, ZX are summed into (dz,dy,dx) and cellprob is also summed. There’s no normalization/weighting per voxel, so when views disagree (mitosis / halo edges / anisotropy), components can cancel and produce weak vectors across a run of z-slices. That makes dynamics under-converge even though the slice looks “red”.

2. Seeding and region growth are “count thresholds” on the end-point histogram After dynamics, seeds are local maxima where count > 10 in the histogram of pixel end-points, and region growth requires h_slc > 2 in local neighborhoods. Thinner cross-sections (typical at the top/bottom of a cell or during mitosis) often fail those discrete counts → no seed / no expansion for that plane, producing exactly the gaps in z slices that I am finding. None of this uses the magnitude of cellprob; it’s just used to gate pixels before dynamics. The flows passed to dynamics are hard-gated and divided by 5: follow_flows(dP * (cellprob > cellprob_threshold) / 5., ...) If fusion has already weakened vectors, this extra /5 is enough to stop convergence in those red high cell probability z planes.

Screenshots

[derekthirstrup]

@mrariden I have found the source of the missing z slices bug. I had to modify the dynamics.py file to tune a few of the variables used for filtering seeds and flow scales. The max size fraction was also dropping z slices. Do you want me to push the modifications that were necessary to fix this bug?

[mrariden]

@derekthirstrup A PR would be very helpful, yes. Please also share some example data for our testing.

Thanks!