lmdeploy
lmdeploy copied to clipboard
[Bug] 0.5.3 回复内容重复性高
Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
Describe the bug
4bit版:https://openxlab.org.cn/apps/detail/yisheng/snh48_4bit
目前尝试的是internlm2.5-7b-chat。然后4bit量化之后,会出现重复说话不停止的问题。
制作conversation对,上下文分别放在input和output中,原本500条数据,增广数据(转写法)最终3000条数据。
相关:https://github.com/InternLM/xtuner/issues/908#issue-2480362048
Reproduction
Environment
*
Error traceback
No response