lightseq icon indicating copy to clipboard operation
lightseq copied to clipboard

Published 20 hours ago • verified publisher icon bytedance

模型转换问题：encode_output_project_kernel_kv

Open 13354236170 opened this issue 3 years ago • 1 comments

  1. 请问模型转换的时候 encode_output_project_kernel_kv 这个参数是来自于什么操作呢？我是使用的torch自带的decoder 构建的模型，没有找到对应的参数转换到这个参数上呢，我看大小要求hidden_sizehidden_size2 *dec_layer_num 。
  2. 请问构建TransformerDecoder 推理引擎的时候有什么办法跳过这个encode_output_project_kernel_kv 操作吗？

13354236170 avatar Jun 02 '22 08:06 13354236170

It's from kv projection in decoder cross attention which should not be skipped.

Taka152 avatar Jun 03 '22 07:06 Taka152