Transfer learning

[Zhw098]

I am newbie of opennmt. The library is very helpful for me. And I have some questions. I want to apply a transfer learning strategy in neural machine translation.

First, translate models using large-scale parallel corpus training. but, How can I get model parameter weight? Then, how do I transfer the weights of the parameters to the new translation framework (opennmt)? Looking forward to your advice or answers. Thanks!

[francoishernandez]

You will have to be more specific.

Then, how do I transfer the weights of the parameters to the new translation framework (opennmt)?

If OpenNMT-py is the "new" translation framework. What is the "old" one?

Anyways, there are not really any such conversion tools for now. You might want to build some custom conversion script, like this one for instance.

Once you have a compatible checkpoint, you can just use the train_from flag to continue training.

[Zhw098]

You will have to be more specific.

Then, how do I transfer the weights of the parameters to the new translation framework (opennmt)?

If OpenNMT-py is the "new" translation framework. What is the "old" one?

Anyways, there are not really any such conversion tools for now. You might want to build some custom conversion script, like this one for instance.

Once you have a compatible checkpoint, you can just use the train_from flag to continue training.

I prepare both the parent model and the child model to use the Transformer model of the OpenNMT framework. Just keep the parameters of the parent model and transfer to the child model, but I don't know how to do it. Thanks

[Zhw098]

train_from

Can you explain the train_from flag in detail? Suppose I already have the parent model(OpenNMT) Thanks!

[francoishernandez]

There is not much to explain. You set the path of the checkpoint you want to continue training from train_from: my_model.pt.

You probably just need to dive in and experiment by yourself at this point.