[BUG]: Outliers in cell segmentation points (both for ['mask'] and ['outlines'])

[postnubilaphoebus]

Describe the bug The outlines as given by Cellpose vary in density, sometimes producing outliers. This becomes problematic when approximating the shape of the cell using the outlines. Is this behaviour intended, and what could it signify when outliers are present in the segmentation? For now, I assume rejecting these outliers is best practice.

Screenshots

[carsen-stringer]

sorry I'm not sure I understand, we use cv2 to make the outlines but they might not be perfect. but there are also mask outliers? this is with do_3D=True or with stitching?

[postnubilaphoebus]

Hello, my apologies for the late reply. I used python -m cellpose --Zstack to run a fine-tuned 3d model. So I assume it has to do with 3D=True. But I can investigate this further if you like. I did not question this observation, because my thinking was that it is inherently difficult to model boundary conditions with heat diffusion (especially in 3d, or by combining 2d heat diffusion outputs in 3d like cellpose does). If you have a different intuition, please let me know.

[carsen-stringer]

for the outlier mask points you could run connected components and use that to create the mesh: https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.label.html, we can also add this as an optional post-processing step

[derekthirstrup]

I have also found extra single pixel objects in 3d masks with previous version of cellpose. I just made a PR for a bug fix to dynamics.py that fixes a tensor dim mismatch index error that inadvertently might have solved this issue. I suspect it is one of these two source that were the root cause of the issue.

1. Convergence to Noise or Irregular Flow Patterns:

Issue: During the dynamics simulation, if the flow fields (dP) contain irregularities or noise, some pixels might not have a clear convergence direction. This ambiguity can cause certain pixels to converge to themselves, resulting in single-pixel seeds.

Cause: Flow Field Quality: Poorly predicted flow fields can have inconsistent directions or magnitudes, leading to unreliable dynamics. Normalization Errors: Incorrect normalization of flow vectors can distort the intended movement of pixels. Edge Effects: Pixels at the boundaries or regions with low cell probability might not follow the flow correctly, leading to convergence anomalies. Impact: These single-pixel seeds can be mistakenly interpreted as individual cells, cluttering the final mask with false positives.

2. Seed Detection Mechanism Flaws:

Issue: The find_objects function identifies regions in the mask by locating connected components. If the flow simulation incorrectly assigns a unique convergence point to a pixel, find_objects might detect it as a separate object.

Cause: Incorrect Mask Labeling: If multiple pixels erroneously converge to the same single pixel, they might not be aggregated correctly. Padding and Indexing Errors: Improper handling of padded regions or incorrect tensor indexing can result in isolated pixels being marked as separate entities.