opencompass icon indicating copy to clipboard operation
opencompass copied to clipboard

[Bug] 使用deepseek-v2-lite-chat进行humaneval测试时,出现问题,无论是在官网还是本地部署测试分数只有1.22,其他模型都正常

Open shuowoshishui opened this issue 9 months ago • 4 comments

Prerequisite

Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

Image

Image

Image

Reproduces the problem - code/configuration sample

Image 使用该配置

Reproduces the problem - command or script

出现的问题

Reproduces the problem - error message

Other information

No response

shuowoshishui avatar Mar 10 '25 10:03 shuowoshishui

其他的模型无论是在线测试还是我在离线测试分数都正常

shuowoshishui avatar Mar 10 '25 10:03 shuowoshishui

Thank you for the report. We will investigate this issue.

tonysy avatar Mar 10 '25 10:03 tonysy

Image 对了我使用evalscope+opencompass推理结果也是比较正常的

shuowoshishui avatar Mar 10 '25 10:03 shuowoshishui

I also find this problem. Plz fix it.

qianweijiujiu avatar Apr 29 '25 05:04 qianweijiujiu