Validation loss keeps increasing

[ccSoVegetable]

Hi,

Thank you for sharing this interesting work. I am trying to use your code to fine tune the GPT2 model with the knowledge sentences you provided, but meet a problem that the validation loss keeps increasing after epoch 1.

Also, when I try fine tuning with the roc stories dataset, the validation loss decreases during the first two or three epoch and increases after that. The enviroment has been set up according to you suggested in the prerequisites. Could you please give some suggestions?