Perplexity value during training remains constant

[Yaffa16]

When i train on the Shakespeare data, for both shakespeare_0 or shakespeare_1, the perplexity value in eval_results remins constant perplexity= tensor(135.8836)

Although the model trains till checkpoint=13797 , the quality of trained output is consistently low .

I also tried to run the demoparaphraser script while pointing to the Shakespeare model directory downloaded from the GoogleDrive here, however the quality was still not the same as on the web demo API .

Can you please help with why is this happening

[martiansideofthemoon]

hi @Yaffa16, Thanks for reporting the issue! Is your training loss going down? This is logged in Tensorboard (https://github.com/martiansideofthemoon/style-transfer-paraphrase/blob/master/style_paraphrase/run_lm_finetuning.py#L245) but you could also print it directly.

Also, what's the logging of the evaluation script? (this loop - https://github.com/martiansideofthemoon/style-transfer-paraphrase/blob/master/style_paraphrase/run_lm_finetuning.py#L444)

It should output the perplexity of every checkpoint.