faster-rnnlm
faster-rnnlm copied to clipboard
yandex
Benchmark on 1 Billion
You report training times in the 1 Billion dataset, but not test perplexity. It would be helpful to compare this implementation with others if you could post the test perplexity and the hyperparameters that could be used to reproduce the result.
Also, thanks for putting this on Github. :)
I will definitely add some ASAP! And could you advice any other implementations that could handle one billion word dataset? I'm awawre about torch-based HS. Everything else seems to be too slow for such a huge vocabulary.
Possibly the only library that comes close to handling it right now is: https://github.com/nicholas-leonard/dp/.
Looking forward to your results.
I've finally added information about finally of different models. Check out https://github.com/yandex/faster-rnnlm#experiments
The benchmarks are really cool! Thanks!
How do you interpolate with the KN-5 model, though? And from where is the KN-5 model obtained?
The model is downloaded from benchmark's repo. I used quite straightforward interpolation, i.e. mean per-word probability, as it's describe in the benchmarks' paper.