(Wikihow-T5) Not seeing any ROUGE Score improvement in training

[cvillela]

Hello!

I am using the wikihow-fine-tuning-T5 script (unchanged, except I'm using tyhe XSUM dataset) for some research, but I'm having trouble training the model.

Although the dataset steps seem to work perfectly, and I'm able to run the training steps with the Trainer() class, my rouge score is stuck at the initial value. There is no improvement, and the computed metrics remain constant throughout the entire training process. Would anyone have any ideas on why this is happening?

I am sending a picture of my WandB logs for this specific run (~40 epochs, few training/val/test examples because I am trying to overfit the model in order to test training performance).

Thanks a lot!

[KotikMatmexa]

Were you able to resolve this issue? I'm having the same problem on a different dataset.