training and validation loss looks funny

[munziliashali]

Dear @meijieru and all,

I am using your model with additional cnn layer on my part to predict my custom dataset and got this funny-looking training loss and validation loss... Can you point out how should I fix this?

Thank you.

[GSWS]

Hi, did you solve your problem? Did you try using a smaller learning rate?

[munziliashali]

Hi, there.

Yes, I solved it by using batchNormalization=True in every CNN layer.. good luck

[zhengshangdong]

@munziliashali Hi,if you use batchNormalization=True,the pretrained model can not load straightly,how you solve this new problem.....Only one way I can come out——rebuild the network and get the convs(weights bias) from pretrained model by keys,but It's too trouble.....

[Heisenberg0391]

@munziliashali Hi,if you use batchNormalization=True,the pretrained model can not load straightly,how you solve this new problem.....Only one way I can come out——rebuild the network and get the convs(weights bias) from pretrained model by keys,but It's too trouble.....

does param 'strict' help? try model.load_state_dict(pretrained_dict, strict=False)