opencompass

opencompass copied to clipboard

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

hailsham

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

bittersweet1999

S3eval branch

5

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

lfy79001

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

bittersweet1999

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand...

yanyc428

在进行MPT-7B模型的评测时运行[Bug]

1

### 先决条件 - [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。 - [X] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。 ### 问题类型 我正在使用官方支持的任务/模型/数据集进行评估。 ### 环境 _torch_pytree._register_pytree_node( {'CUDA available': True, 'CUDA_HOME': '/usr/local/cuda', 'GCC': 'gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0',...

ColinHuanghb

### Describe the feature As proposed in https://github.com/InternLM/opencompass/issues/80#issuecomment-1644917593 ### Will you implement it? - [ ] I would like to implement this feature and create a PR!

Leymore

### 描述该错误 https://opencompass.readthedocs.io/en/latest/advanced_guides/new_model.html 这个文档中已经有关于如何新增model支持的描述，但是对于如何完成评测一个新增model，这部分文档还比较分散，比如需要： 1. 新增model类的几个方法的实现（以api为例） 2. 新增对应config 希望文档能有一个整体性的描述，对小白用户比较友好 ### 环境信息 无 ### 其他信息 _No response_

del-zhenwu

[Bug] Partition tasks sometimes fail due to occupied ports

1

Hi, thanks for sharing this great open-source project! When using multiple GPUs for evaluation, I found partition tasks sometimes will fail due to occupied ports. ### Prerequisite - [X] I...

sdc17

### Describe the feature Hi, thanks for sharing this wonderful project! I am wondering is there somewhere we can find the config version that is used to produce the score...

sdc17