neuralmonkey icon indicating copy to clipboard operation
neuralmonkey copied to clipboard

NematusGRU initialization

Open kocmitom opened this issue 7 years ago • 0 comments

I examined Nematus code: https://github.com/EdinburghNLP/nematus/blob/master/nematus/layers.py

which should be the same as ours: https://github.com/ufal/neuralmonkey/blob/master/neuralmonkey/nn/ortho_gru_cell.py#L57

They use ortho_weight and norm_weight. I did not go into details if ortho_weight is equal to our orthogonal initialization. But we are using tf.glorot_normal_initializer() but they are using: https://github.com/lium-lst/nmtpy/blob/master/nmtpy/nmtutils.py#L66

0.01 * np.random.randn(nin, nout)

kocmitom avatar Apr 03 '18 16:04 kocmitom