opencompass

opencompass copied to clipboard

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

disperaller

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

macheng6

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

bullw

[Bug] hf_chatglm3_6b评测AFQMC数据集时，自测结果与官方不一致。且自测结果不稳定。

1

### 先决条件 - [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。 - [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。 ### 问题类型 我正在使用官方支持的任务/模型/数据集进行评估。 ### 环境 ``` {'CUDA available': True, 'CUDA_HOME': '/usr/local/cuda-12.3', 'GCC': 'x86_64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0',...

jieheroli

Integrate lmdeploy pipeline api

1

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

lvhan028

### Describe the feature (opencompass) -bash-4.2$ python run.py ./configs/api_examples/eval_api_zhipu_v2.py --debug Traceback (most recent call last): File "/home/wumao/opencompass-main/run.py", line 4, in main() File "/home/wumao/opencompass-main/opencompass/cli/main.py", line 225, in main cfg = get_config_from_arg(args)...

1518630367

[Feature] 支持openai/GPT4-o的评测seting

1

### 描述该功能 链接： https://github.com/openai/simple-evals GPT4-o的这套评测seting是非常具备参考价值的。预计被大家关注并使用。 ### 是否希望自己实现该功能？ - [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！

Ezra-Yu

### 描述该功能 Leval数据集希望新增两个config：codeU和sci_fi ### 是否希望自己实现该功能？ - [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！

liuxize-dhu

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

SefaZeng

### Describe the feature If you want to change the post-processing or evaluation logic now, you must delete the results file manually, need add -f to automate this process. ###...

liushz