opencompass
opencompass copied to clipboard
Add GrandPhysics dataset
Motivation
This PR adds support for the GrandPhysics dataset to OpenCompass. GrandPhysics is a evaluation dataset (not yet publicly available) built from A Grand Dictionary of Physics Problems and Solutions (物理学大题典). Dataset file need to be uploaded to OSS. GrandPhysics_50_questions.json
Modification
-
Dataset file: opencompass/datasets/grand_physics.py
-
Dataset evaluation configuration: opencompass/configs/datasets/GrandPhysics/GrandPhysics_llm_judge_gen.py
Checklist
Before PR:
- [x] Pre-commit or other linting tools are used to fix the potential lint issues.
- [ ] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
- [ ] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
- [ ] The documentation has been modified accordingly, like docstring or example tutorials.
After PR:
- [ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
- [ ] CLA has been signed and all committers have signed the CLA in this PR.