Amir M.

Navigate Back

1

Hi When following internal links in PDFs, e.g., looking at a reference, there is a great need to be able to come back to the previous location and continue reading...

Add FreebaseQA to tasks and gauntlet

2

Added [FreebaseQA](https://aclanthology.org/N19-1028/) which is used in several recent papers [1](https://link.springer.com/chapter/10.1007/978-3-031-44216-2_28) [2](https://arxiv.org/abs/2403.09712) [3](https://openreview.net/forum?id=B9klVS7Ddk). Tested on [Vicuna-7B](https://huggingface.co/lmsys/vicuna-7b-v1.3) and obtained similar results to [3](https://openreview.net/forum?id=B9klVS7Ddk). Feel free to cherry-pick if we only need the...

A similar feature to the [library usage of lm-eval-harness](https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/interface.md) using a simple registry add-on.