mos

mos copied to clipboard

cannot reproduce result?

6

Hi, When I try to reproduce the results, I find that the model converges ~100 epoch and the valid ppl is 65.32, which is much higher than the published results....

haoyudong-97

Hi Mr.Dai, It seems the performance discrepancy has lead to slightly worse performance on the penn treebank in the readme page. I am able to reproduce the result for the...

tonytan48

Hi, I was trying to run MoS on WikiText-103 and 1B word dataset. I wonder if you have used adaptive softmax, such as this paper **_Efficient Softmax Approximation for GPUs_**,...

stylelohan

Update main.py

2

the current version of ``--cuda`` is store_false, maybe it would be better to change it

LiyuanLucasLiu

Hi! Thanks for sharing this code base! Do you have a pre-trained model that we could use? We want to test the idea of incorporating a language model into our...

ziyaoh

I am running this code on my Linux server, however it seems that this python program is only runned by one of my computer's cpu and leaves two thirds total...

zml24