Bleu scores

[iliasprc]

When i run the inference with the trained model with the evaluation ,the train and the test set i get the following BLEU-4 scores Train set: 2.0 Dev set: 8.4 Test set: 9.3 How is that possible to get lower scores using the training set?

[neccam]

Hi,

This is quite interesting and have never encountered myself. Would you mind sharing the commands you have used?

[iliasprc]

I have used the following command for the inference model python -m nmt --out_dir=trained_model --inference_input_file=Data/phoenix2014T.test.sign --inference_output_file=Data/predictions_test.de --vocab_prefix= Data/phoenix2014T.vocab --inference_ref_file=Data/phoenix2014T.test.de --base_gpu=0

where every time i change the inference input and ref files to train, test , dev

[neccam]

Thanks a lot, I will investigate to see what went wrong with it.

[MeteorsHub]

Same issue. The scores on my machine are Train set: 1.8 Dev set: 9.3 Test set: 8.5