Broken training script

[bocchilorenzo]

Hi, I was trying to use the training script, but it appears to be broken. More specifically, for any language but romanian it doesn't even start the training, and with romanian it doesn't do more than 1 iteration both in the training itself and in the subsequent evaluation. Also, the import at line 4 is wrong. After removing the lines 22, 23, 46, 47, 81 and 82 from the script, and changing line 4 from "from eurovoc_transformers.model import LangModelWithDense" to "from pyeurovoc.model import LangModelWithDense", everything seems to work correctly. I didn't fully train it yet, so I can't confirm that it goes till the end without issues. But, for 500 iterations it worked fine.

[ziorufus]

Hello! I am trying to run the code, too. Here you can find how @bocchilorenzo modified the code: https://github.com/racai-ai/pyeurovoc/compare/master...bocchilorenzo:pyeurovoc:master (you can ignore the part where the BERT model has been changed)

In particular, the weird thing is that during training F1 seems ok, but when the (training) evaluation starts it gives values over 1 (we tried by using dbmdz/bert-base-italian-cased model).

This is the log:

Epoch: 1/5
        Training - It: 0, Loss: 0.0000, F1: 0.0000
        Training - It: 100, Loss: 0.6842, F1: 0.1533
...
        Training - It: 900, Loss: 0.2569, F1: 0.1701
        Training - It: 1000, Loss: 0.2396, F1: 0.1632
        Evaluating - It: 0, Loss: 0.0000, F1: 0.0000
        Evaluating - It: 100, Loss: 0.0181, F1: 3.6604

        New best model found: -1.0000 -> 3.7820

I think that it's not normal to have F1 = 3.7.

Can you tell us whether our modifications on the code are correct? Or, alternatively, can you send to us a version of the code that works?

[ziorufus]

I tried to use many more epochs, and now F1 seems realistic (more than 50). So the only question is: are our modifications to the code correct?