RNN results not reproducible

[Wronskia]

Hello, Thanks for open sourcing the code. After your commit: https://github.com/melodyguan/enas/commit/2734eb2657847f090e1bc5c51c2b9cbf0be51887

I get 63.26 in ppl and not the 55.6 stated in the paper. However before this commit I get 55.6. Is there something I am missing?

Thanks

[hyhieu]

Hi,

Thanks for your interest. Commit 2734eb2 fixed a bug in the evaluation process. After the fix, we had to further tune the model's hyper-parameters to reach a good performance. The best number we could reach was 56.3. We have updated the paper and will soon role out a commit to fix the bug in the code. We apologize for the mistake.

[liamcli]

Could you update the ptb_final.sh script to use the architecture and hyperparameters you used to get to the 56.3 perplexity reported in the paper? Thanks!