Lucas Weber
Results
2
issues of
Lucas Weber
There were problems with multi-machine runs due to the use of `LOCAL_RANK` instead of `RANK` for assigning tasks to devices (see [#201](https://github.com/facebookresearch/llama/issues/201)). With this fix, the models should be usable...
CLA Signed
This PR adds *tinyBenchmarks* ([paper](https://arxiv.org/abs/2402.14992)) to the eval harness. *tinyBenchmarks* currently already uses eval-harness for evaluation (see dataset-cards; e.g. [tinyHellaswag](https://huggingface.co/datasets/tinyBenchmarks/tinyHellaswag)), but requires users to create their own configs.