opencompass
opencompass copied to clipboard
[Bug] 使用deepseek-v2-lite-chat进行humaneval测试时,出现问题,无论是在官网还是本地部署测试分数只有1.22,其他模型都正常
Prerequisite
- [x] I have searched Issues and Discussions but cannot get the expected help.
- [x] The bug has not been fixed in the latest version.
Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
Reproduces the problem - code/configuration sample
使用该配置
Reproduces the problem - command or script
出现的问题
Reproduces the problem - error message
。
Other information
No response
其他的模型无论是在线测试还是我在离线测试分数都正常
Thank you for the report. We will investigate this issue.
对了我使用evalscope+opencompass推理结果也是比较正常的
I also find this problem. Plz fix it.