Presence of low counts squares

[guigir]

I've used proseg to segment several xenium datasets and each time I see squares in the distribution of transcripts and genes. These seem to follow the fov boundaries but I don't understand why cells along these fov have particularly low counts. These structures are not present with the initial Xenium segmentation.

Here are some plots showing the issue where you can see horizontal and vertical lines for the proseg segmentation

Do you have any idea where this might be coming from?

[dcjones]

I don't know exactly what's happening here, but there can be fov effects which can manifest as reduced detection of transcripts along the edges of a fov. Because proseg is inferring cell morphologies based on transcripts, its segmentation may be more sensitive to this phenomenon. Hopefully, this affects a relatively small proportion the cells though. As of yet, I don't really have a good plan to try model or counter this.