Unable to finetune

[clararodrigo]

Hello, I am trying to further finetune MedSAM2 as I have quite a niche imaging application. I have set the model to training mode by doing: predictor.train(True) predictor.sam_mask_decoder.train() However, the loss never goes down and when I print out the gradient of all weights in the model, all are None. Of course, I am doing: optimiser.zero_grad() loss.backward() optimiser.step()

I would greatly appreciate any help to make my model learn, thanks in advance.

Clara