Perry Li

Same question, I also wonder how to get the cache files.

Same question, I also want to use lightseq to accelerate MarianMTModel.

> Is it a standard Transformer model? If it is, we can write an export script to export from Marian model to LightSeq model, and then infer without any modification....

Lightseq doesn't support MarianMTModel. https://github.com/bytedance/lightseq/issues/422 @dengcunqin @qiubinyang

Here is my self export script, it's basicly same with `hf_bart_export.py` ```py """ Export Hugging Face BART models to protobuf/hdf5 format. """ import os from collections import OrderedDict import tensorflow...

I guess this error occurred at [`transformer_weight.cc`](https://github.com/bytedance/lightseq/blob/master/lightseq/inference/proto/transformer_weight.cc#L205) and the reason is that the `_hidden_size` is initialized by [`_hidden_size = transformer.trg_embedding().norm_scale_size()`](https://github.com/bytedance/lightseq/blob/master/lightseq/inference/proto/transformer_weight.cc#L36). However MarianMTModel doesn't have this module. Do I have any...

@zjersey Sorry, I might make a mistake previously. In huggingface,`BartForConditionalGeneration` model's en(de)coder has two module which is called `encoder.layernorm_embedding` and `decoder.layernorm_embedding`. As [huggingface's source code](https://github.com/huggingface/transformers/blob/main/src/transformers/models/bart/modeling_bart.py#L808) shown, the two modules are...

> @zjersey Sorry, I might make a mistake previously. In huggingface,`BartForConditionalGeneration` model's en(de)coder has two module which is called `encoder.layernorm_embedding` and `decoder.layernorm_embedding`. As [huggingface's source code](https://github.com/huggingface/transformers/blob/main/src/transformers/models/bart/modeling_bart.py#L808) shown, the two modules...

> @Youggls For now this model is not supported. > > For BART, `layernorm_embedding` operates in [self-attention of the first layer](https://github.com/bytedance/lightseq/blob/master/lightseq/inference/model/encoder.cc.cu#L170), while the kernel `ker_norm_layer_resual_launcher` doesn't support layernorm pointers to...

非常感谢您的解释和分享！祝您科研工作顺利~