opencompass

opencompass copied to clipboard

[Feature] 为啥我开始评测一直卡在这里

1

### Describe the feature  ### Will you implement it? - [ ] I would like to implement this feature and create a PR!

GEK1

[Bug] When I attempted to perform the agent evaluation, the console returned an error: "AttributeError: 'OpenAI' object has no attribute 'chat'".

1

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

CaptainJi

### Describe the feature ``` 06/18 23:09:38 - OpenCompass - INFO - Task [Deepseek/alignment_bench] 06/18 23:09:38 - OpenCompass - DEBUG - Modules of opencompass's model registry have been automatically imported...

Chensem

### 先决条件 - [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。 - [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。 ### 问题类型 我正在使用官方支持的任务/模型/数据集进行评估。 ### 环境 python run.py --models hf_llama3_8b --datasets ARC_c_ppl ### 重现问题 - 代码/配置示例 无...

linboyang

opencompass公开榜单更新[Feature]

2

### 描述该功能 之前提到说上周会对opencompass的公开评测集榜单进行更新，请问现在有更新吗，能否发下链接。想参考一下评测结果 ### 是否希望自己实现该功能？ - [ ] 我希望自己来实现这一功能，并向 OpenCompass 贡献代码！

cobraheleah

[Feature] 如何在needlebench 中使用api model？

2

### Describe the feature 如何再needlebench 中使用api model，api example中的用法用不了 ### Will you implement it? - [ ] I would like to implement this feature and create a PR!

1518630367

[Feature] Difficulty in Evaluating Custom Models with OpenCompass

1

### Describe the feature Dear OpenCompass Team, I've encountered a challenge with OpenCompass when trying to evaluate a custom model that I developed. Currently, it seems that any action I...

jiangjiadi

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

chairmanQi

[Bug] llm-compression task faild at eval stage with latest version

3

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

mqy004

### Prerequisite - [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expected help. - [X] The bug has not been fixed in the [latest version](https://github.com/open-compass/opencompass). ### Type...

chairmanQi