about inference

[piaohe20221128]

hi，thankyou for release code！ I have a question about the different pipline between train and inference 。the paper says that in inference stage the predict out of every decoder layer is fed to the next layer 。But in train stage there need to have embedding 、concat and linear op to generate new tensor which is fed to the next layer 。What would be the impact of this operation?

[chenyangh]

Hi. Both the training and inference use the concatenation of embedding and the original hidden state as the input to the next layers. They should be the same.

[piaohe20221128]

thankyou ！