xFasterTransformer qwen1.5-32b long text input issue

qwen1.5-32b long text input issue

Open zhm-algo opened this issue 1 year ago • 2 comments

May 21 '24 01:05 zhm-algo

could you pls give more details with examples?

May 23 '24 02:05 pujiang2018

在对 32B 进行了错误边缘检测后发现，当模型的输入文本大小大于 1.5K 以上就会出现，生成异常的问题。如果输入文本大小在 1.5k-4k 这个区间内，会不断地循环输出错误内容。如果输入文本大小大于 4K 时，模型响应结果是不断地输出回车符。

输入使用prompt.json 文件中 qwen 对应的8192/4096，可以复现问题 https://github.com/intel/xFasterTransformer/blob/main/benchmark/prompt.json

May 23 '24 03:05 zhm-algo

@marvin-Yu How about this issue, any finding?

Jun 06 '24 04:06 pujiang2018

The latest version does not reproduce the issue， @zhm-algo you can try the latest version again.

Jun 06 '24 04:06 marvin-Yu

Let's close since 2 weeks passed. @zhm-algo pls reopen it if the issue is still there.

Jun 20 '24 01:06 pujiang2018