PreSumm
PreSumm copied to clipboard
🐛`-test_all` does not actually test all checkpoints
When testing checkpoints with -mode validate
and -test_all
options, I expect the code to run the validation on every checkpoint.
However, sometimes it silently stops after evaluating X checkpoints. I believe it's from this code :
https://github.com/nlpyang/PreSumm/blob/8ad9dd0dbb18d92f3d6fa771696ab8b53778954f/src/train_extractive.py#L117-L118
Is it early-stopping ? (If there is no improvement after 10 checkpoints, remaining checkpoints are not evaluated)
If so, it might be better to let the user the choice and put this as a parameter.
Have you solved this problem?
I changed the hard-coded value (10
) to something much higher..