About Shared Embedding

[dawei-yu]

if False:
    model.src_embed[0].lut.weight = model.tgt_embeddings[0].lut.weight
    model.generator.lut.weight = model.tgt_embed[0].lut.weight

Hi, I can't find tgt_embeddings in your code. Maybe it is model.src_embed[0].lut.weight = model.tgt_embed[0].lut.weight.

And if shared Embedding, Should the code be add to make_model()？ Like

def make_model(src_vocab, tgt_vocab, N=6, d_model=512, d_ff=2048, h=8, dropout=0.1):
    ...
    model = EncoderDecoder(...)
    model.src_embed[0].lut.weight = model.tgt_embed[0].lut.weight
    model.generator.lut.weight = model.tgt_embed[0].lut.weight
    ...
    return model

[wangjunhe8127]

i have the same question

[gitfourteen]

It is a typo and it is a False condition. Or else you can try BPE.