opencompass

opencompass copied to clipboard

add daily test. can run more models and datasets. if the score is not null and between baseline * 0.97 and baseline * 1.03, return true. ps, use newest pytorch...

zhulinJulia24

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

hailsham

### 先决条件 - [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。 - [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。 ### 问题类型 我正在使用官方支持的任务/模型/数据集进行评估。 ### 环境 * ### 重现问题 - 代码/配置示例 from mmengine.config import read_base with read_base():...

yangyang-PKU

[Bug] Error occurred when running the BBH dataset

1

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

niexufei

### 先决条件 - [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。 - [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。 ### 问题类型 我正在使用官方支持的任务/模型/数据集进行评估。 ### 环境 - ### 重现问题 - 代码/配置示例 [[-](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606)](https://github.com/open-compass/opencompass/actions/runs/7705226293/job/20998812606) ### 重现问题 - 命令或脚本 -...

del-zhenwu

### 先决条件 - [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。 - [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。 ### 问题类型 我正在使用官方支持的任务/模型/数据集进行评估。 ### 环境 python -c "import opencompass.utils;import pprint;pprint.pprint(dict(opencompass.utils.collect_env()))" {'CUDA available': True, 'CUDA_HOME': '/usr/local/cuda', 'GCC':...

HelanHu

## Motivation Support NPHardEval Dataset for complex reasoning ## Modification Add dataset configs, preprocess codes, and answer validity check codes for NPHardEval dataset. ## BC-breaking (Optional) Does the modification introduce...

Skyfall-xzz

[Bug]

1

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

mengrusun

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

TousenKaname

Need to wait for service testing before merging ## Motivation Please describe the motivation of this PR and the goal you want to achieve through this PR. ## Modification Please...

Connor-Shen