Finetune SAM on rgb dataset (cityscapes)

[alanent]

Thank you for your work. I manage to reproduce your implementation but I encounter problems adapting it to the dataset: https://www.cityscapes-dataset.com/ Do you have a recommendation to adapt your implementation to this dataset ?

[guobabaya]

Have you solved this problem? I also want to apply the model to other data sets. Do you have any experience?

[yao-yu1999]

谢谢你的工作。我设法重现您的实现，但在将其适应数据集时遇到问题：https://www.cityscapes-dataset.com/ 您是否建议将实现调整到此数据集？ me too

[hitachinsk]

Hello, thanks for your interest in our work. Since medical images only contain ONE channel, I duplicate the channel dimension of the medical images from 1 to 3. Maybe you should remove this operation when fine-tuning on the rgb dataset.

[alanent]

I tested this approach by replicating your preprocessing but normalizing by the settings recommended by the SAM team. The results are very bad. Do you have any other modification suggestions? thanks again

Le sam. 17 juin 2023, 15:22, Kaidong Zhang @.***> a écrit :

Hello, thanks for your interest in our work. Since medical images only contain ONE channel, I duplicate the channel dimension of the medical images from 1 to 3. Maybe you should remove this operation when fine-tuning on the rgb dataset.

— Reply to this email directly, view it on GitHub https://github.com/hitachinsk/SAMed/issues/12#issuecomment-1595759927, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH4ANRJQUPUNIM53HXE45VTXLWVTBANCNFSM6AAAAAAZF4PLHQ . You are receiving this because you authored the thread.Message ID: @.***>

[yao-yu1999]

我通过复制您的预处理测试了这种方法，但是 通过 SAM 团队建议的设置进行规范化。结果 非常糟糕。您还有其他修改建议吗？谢谢 再 勒萨姆。17 juin 2023， 15：22， 张开东 @.> 埃克里特 ： ... Hello, thanks for your interest in our work. Since medical images only contain ONE channel, I duplicate the channel dimension of the medical images from 1 to 3. Maybe you should remove this operation when fine-tuning on the rgb dataset. — Reply to this email directly, view it on GitHub <#12 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH4ANRJQUPUNIM53HXE45VTXLWVTBANCNFSM6AAAAAAZF4PLHQ . You are receiving this because you authored the thread.Message ID: @.> Hello. May I ask how you use the network branch that joined LoRA, and then fine-tune it to your own dataset? I changed "num_class=10" (the original code is 8), and used "vit_b", so that the embedding can be divisible, I changed num_heads to 16 (the original code was 12), but an error was reported.

[iamshrey22]

I modified the SAMed code for RGB images. In my case, the images were of corroded metals. Here is the link:

https://github.com/iamshrey22/CorrSAM-Segment-Anything-Model-with-LoRA-Fine-Tuning-for-RGB-images