overfitting

[jbdel]

Hello,

Thank you for this project, looks great ! However i faced overfitting with binary=False (have not tried binary=True)

2019-11-22 15:36:56.690 | INFO     | bert_sentiment.train:train:86  - epoch=1
2019-11-22 13:27:47.669 | INFO     | bert_sentiment.train:train:88 - train_loss=0.0481, val_loss=0.0378, test_loss=0.0370
2019-11-22 13:27:47.669 | INFO     | bert_sentiment.train:train:91 - train_acc=0.299, val_acc=0.494, test_acc=0.504

.....

2019-11-22 15:36:56.690 | INFO     | bert_sentiment.train:train:86 - epoch=29
2019-11-22 15:36:56.690 | INFO     | bert_sentiment.train:train:88 - train_loss=0.0010, val_loss=0.0916, test_loss=0.0854
2019-11-22 15:36:56.690 | INFO     | bert_sentiment.train:train:91 - train_acc=0.991, val_acc=0.500, test_acc=0.533
2019-11-22 15:36:56.690 | SUCCESS  | bert_sentiment.train:train:98 - Done!

Cant seem to improve on valset and testset.

Maybe need other config as stated in here

[munikarmanish]

Hi @jbdel, thanks for your interest in this code.

To be honest, I never ran it for a full 30 epochs. I got the best results in about 5~10 epochs and stopped the program. I have not implemented automatic early-stopping. Did you check the results after every epoch? I don't have GPU access to run it right now. I'll test it again tomorrow.

[KingS770234358]

Hello: May I ask you how did you set your learning rate? My program set learning rate to 0.00001.But I seemed that the loss never decreased @munikarmanish

[KingS770234358]

Hello: May I ask you how did you set your learning rate? My program set learning rate to 0.00001.But I seemed that the loss never decreased @jbdel

[jbdel]

Hello,

I havent changed any hyper parameters. As you can see in my logs, my train loss is decreasing but not the others, that is why i was thinking of overfitting