One Billion Word Benchmark precision

[alexnanchen]

Hello,

First thanks for this nice benchmark!

I have a question regarding the last figure. For the "gru-insyn+direct+KN" results, how do you combine KN with gru-insyn+direct? Is it during training? Do you combine log probabilities of the two models during evaluation with some kind of interpolation?

Many thanks!

Alexandre Nanchen