proseg icon indicating copy to clipboard operation
proseg copied to clipboard

Impact of --schedule parameter on final segmentation results

Open PietroAndrei opened this issue 9 months ago • 4 comments

Hi,

I was wondering if you have ever tested how the number of iterations set through the --schedule parameter might affect the final proseg output. I noticed that on my CosMx samples, by keeping the default 150,150,300 setting for --schedule, proseg tends to define a lower number of cells compared to the original number produced by CosMx (In one case proseg found ~90k cells starting from 150K). I don't know if by increasing the number of iterations I might get a different behaviour.

Thanks,

Pietro

PietroAndrei avatar Mar 10 '25 17:03 PietroAndrei

Proseg can lose cells like this if they are very small and smashed together so that they can't be represented with voxels. That's usually a pretty minor effect, though. I've never seen it lose this many. What I would try is running at a higher resolution. I would start with setting --initial-voxel-size 3 (the default is 4) and see if that helps at all.

dcjones avatar Mar 10 '25 17:03 dcjones

I tried to set --initial-voxel-size 3 but it didn't change much (I gained less than 200 cells). This is a representative FOV from the CosMx sample with the most striking difference between CosMx and proseg cell number:

Image

I haven't inspected the cell polygons produced by proseg yet, but to me it does not look like a 'messy' histology. This is the original CosMx segmentation for comparison:

Image

The low cell number might be due to the overall transcript detection quality, although the quality was not particularly different between this sample and the other included in the same CosMx run (for which I didn't get a massive drop in cell number).

PietroAndrei avatar Mar 11 '25 11:03 PietroAndrei

That's interesting. I don't see anything remarkable about this sample that would cause this issue. Another thought that occurred to me is that there may be an issue with the nuclear stain. Proseg is initialized on nuclei, so it's possible CosMx segmentation worked on the cell boundaries but did poorly on the nuclei causing proseg to undercount cells downstream. I would try running with --use-cell-initialization to see if that does better.

dcjones avatar Mar 11 '25 22:03 dcjones

--use-cell-initialization seems to solve the discrepancy, thanks!

PietroAndrei avatar Mar 13 '25 14:03 PietroAndrei