mehta-lab
[Question] Is deconvolution required for inference with pre-trained models?
Hi, I have a question regarding inference using the pre-trained models from your paper, "Robust virtual staining of landmark organelles with Cytoland" (Liu et al., 2025). Is deconvolution (e.g., using waveOrder to get "phase density" ) a mandatory preprocessing step for one's own bright-field images before feeding them into the pre-trained models? The paper states that models were trained on deconvolved data. My understanding is that the models have likely never seen raw bright-field data, so I wanted to confirm if providing non-deconvolved images at inference time would lead to poor results. Thank you!
Hi @Jun-Dele the 2D model (VSCyto2D) was pre-trained with diverse datasets (including brightfield, DIC, Phase). I suggest you start with that and see if you are able to find cells in your data. You're welcome to post the images here for discussion.
https://huggingface.co/spaces/chanzuckerberg/Cytoland
@mattersoflight Thanks for the quick clarification. So, just to further confirm: the pre-trained model works well on raw BF/DIC and deconvolved images because it was trained on a mixed dataset of all of them. Is that correct? And did you consider the proportions between these different data types during training? I will test this using my own wide-field bright-field images (100x objective, NA \approx 1.45) and will update you with my prediction results and findings. Thank you!
Can the VSCyto2D model take a raw 2D bright-field image as direct input, or must I first acquire a 3D z-stack and deconvolve it (e.g., using waveOrder) to get "phase density"?
A previous answer suggested VSCyto2D was trained on "diverse datasets (including brightfield...)."
This suggests the pre-trained VSCyto2D model might accept raw 2D bright-field images, but this is not explicitly stated.
Could you please confirm: Was the VSCyto2D model also trained on raw 2D bright-field images, making the 3D deconvolution step optional for 2D inference?
As acquiring a 3D z-stack just for 2D inference is a significant hurdle, so this will determine if we can use the model, Thanks
- All our models use the FCMAE pretraining and then virtual staining finetuning step. For the first step, yes, we take images from different contrasts( i.e PhC, QPI, BF,etc).
- If you see figure 2 and the task we are trying to regress to deconvolution improves the results.
- You can still make it work with 2D and finetune on your data. If you are interested in doing 2D Phase reconstructions from short stack for ~20x magnification data, then checkout out waveorder
@talonchandler not sure if we have a 2D phase recon example
For a 2D reconstruction from a short brightfield stack, here is an scripted example.
If you're using the CLI, you can use this config with reconstruction_dimension: 2.
Good luck, and let us know how you make out @Jun-Dele.
