Jungwhan Kim

Hi, I think there is an issue with the DROP in current implementation. The scores for the DROP task are nearly zero, and after checking the prompt, it seems that...

keep new line for task description

5

related: https://github.com/EleutherAI/lm-evaluation-harness/issues/1817 I found out that `apply_template` to description removes line break since jinja2. For example, in `mmlu_anatomy` task, yaml file is as below. ```yaml "dataset_name": "anatomy" "description": "The following...