PreSumm icon indicating copy to clipboard operation
PreSumm copied to clipboard

🐛`-test_all` does not actually test all checkpoints

Open astariul opened this issue 5 years ago • 2 comments

When testing checkpoints with -mode validate and -test_all options, I expect the code to run the validation on every checkpoint.

However, sometimes it silently stops after evaluating X checkpoints. I believe it's from this code :

https://github.com/nlpyang/PreSumm/blob/8ad9dd0dbb18d92f3d6fa771696ab8b53778954f/src/train_extractive.py#L117-L118

Is it early-stopping ? (If there is no improvement after 10 checkpoints, remaining checkpoints are not evaluated)

If so, it might be better to let the user the choice and put this as a parameter.

astariul avatar Aug 30 '19 08:08 astariul

Have you solved this problem?

zyw12358 avatar Apr 20 '22 07:04 zyw12358

I changed the hard-coded value (10) to something much higher..

astariul avatar Apr 28 '22 13:04 astariul