opencompass

opencompass copied to clipboard

### Describe the feature Check https://github.com/open-compass/opencompass/wiki ### Will you implement it? - [ ] I would like to implement this feature and create a PR!

zhimin-z

[Feature] Support pass@1 evaluation for multi predictions in MathEvaluator

3

solving issue：https://github.com/open-compass/opencompass/issues/2252

DELEnomore

[Bug] SimpleQA测评报错LMEvaluator._init_()缺失参数'judge_cfg'和'output_path'

2

### Prerequisite - [x] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [x] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

htx113

[Bug] MMLU 数据集在A100上评测运行日志显示某个批次的最后一个运行时就卡住不动，卡死了

1

### Prerequisite - [x] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [x] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

yanchenmochen

[Feature] Addition of BioML-Bench

1

### Describe the feature An interesting new bio benchmark that might be worth adding: https://github.com/science-machine/biomlbench ### Will you implement it? - [ ] I would like to implement this feature...

TKassis

[Bug] 官网的指导手册有更新嘛？和现在master的代码完全对不上，新版本的内容多久能更新，官网对应的版本是多少

3

### 先决条件 - [x] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。 - [x] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。 ### 问题类型 我正在使用官方支持的任务/模型/数据集进行评估。 ### 环境 RT. 忍不住想问问 ### 重现问题 - 代码/配置示例 无 ### 重现问题 - 命令或脚本...

luo877280

solving issue： #2270

Nexround

### Prerequisite - [x] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [x] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

Nexround

### Prerequisite - [x] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [x] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

MichaelYang-lyx

[Bug] Can NOT find valid BAAI/TACO dataset

1

### Prerequisite - [x] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [x] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

ShaneTian