lightseq icon indicating copy to clipboard operation
lightseq copied to clipboard
verified publisher icon bytedance

Is there any plan to support MarianMTModel？

Open dengcunqin opened this issue 3 years ago • 5 comments

Is there any plan to support MarianMTModel？here,https://huggingface.co/Helsinki-NLP/opus-mt-it-en

dengcunqin avatar Sep 19 '22 16:09 dengcunqin

Is it a standard Transformer model? If it is, we can write an export script to export from Marian model to LightSeq model, and then infer without any modification.

godweiyang avatar Sep 26 '22 08:09 godweiyang

Is it a standard Transformer model? If it is, we can write an export script to export from Marian model to LightSeq model, and then infer without any modification.

Yes, looking forward to this script.

qiubinyang avatar Oct 26 '22 20:10 qiubinyang

Same question, I also want to use lightseq to accelerate MarianMTModel.

Youggls avatar Nov 18 '22 02:11 Youggls

Is it a standard Transformer model? If it is, we can write an export script to export from Marian model to LightSeq model, and then infer without any modification.

MarianMTModel's architecture is basically same with BartForContionalGeneration. However it use swish activation instead of gelu.

Youggls avatar Nov 18 '22 02:11 Youggls

Lightseq doesn't support MarianMTModel. https://github.com/bytedance/lightseq/issues/422 @dengcunqin @qiubinyang

Youggls avatar Nov 21 '22 09:11 Youggls