Scores from epoch with best dev-scores:

[ghost]

Hi,

I am just trying to understand what does that sentence mean, "Scores from epoch with best dev-scores:"?

It is reporting score of test and dev set for each epoch, then what does "best dev score" imply? Does it have to do with mini-batch?

[nreimers]

Hi, the development and test score is computed every epoch.

The system further remembers what the epoch with the best development score was. It shows than in every epoch also the dev & test score from the epoch that had so far the best development score.

If the current epoch has the best dev score, it shows the dev & test score from that epoch.

[ghost]

Hi thanks,

Are the actual predictions made by classifier written somewhere?

[nreimers]

The actual predictions are not written to disk during training. However, when you safe the model you can load it and use it for predictions. The git contains several examples how to load the models. For example, see the RunModel_CoNLL_Format.py

If you would like to have the predictions during training, you would need to modify the BiLSTM.fit function.

[ghost]

So i need the function to write the predictions made for dev and test set for each epoch written to the respective files. Only BiLSTM.fit needs to be changed for that?

[nreimers]

Yes. BiLSTM.fit predicts the labels for the dev/test set. You just need to add lines so that these predictions (and maybe the gold labels) are stored in a file.