Songyang Zhang comments

Results 223 comments of


                                            Songyang Zhang

[Bug] DeepSeek R1 32B 模型测评 AIME2024 数据集得分低

We have provided an example on how to re-implement the AIME for DeepSeek-R1-32B. Please check: https://github.com/open-compass/opencompass/blob/main/docs/en/user_guides/deepseek_r1.md

[Bug] The conflict between `datasets>=2.20.0` and `pandas<2.0.0`

Thanks for the report and we will fix this issue soon.

Mmlu pro biomed

It is recommended that you refrain from modifying the default configuration. Instead, you may create a new configuration tailored to your specific requirements.

hle biomed test

It is recommended that you refrain from modifying the default configuration. Instead, you may create a new configuration tailored to your specific requirements.

[Bug] Map with add_length will cause memory OOM

Yes, it's the issue of Arrow Dataset

Fix mkdir hardcode in `runners` and `partitioners`

Thanks for your contribution. Please fix the conflicts.

Process failed

OpenCompass is not compatible with Windows currently

Process failed

Thanks, we will update the README recently.

[Bug] commonsense_qa和strategyqa的results为空

Please check the log/eval and the results folder

[Feature] Support BigCodeBench

Thanks for the suggestions, would you like to contribute this bench into OpenCompass?