instruct-eval
instruct-eval copied to clipboard
declare-lab
This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.
Hi all, I read the code and realized that the results were obtained from 3-shot demonstrations. However, some models were trained to follow instructions without demonstrations. These models may have...
Hi, I noticed that you're using lm_eval v0.2.0 as an evaluation flow for more tasks, however, in v0.4.0 and later lm_eval added more datasets and made major changes to the...
Hello, Is there a way to evaluated a model that we have trained an e.g. LoRA adapter on? Thanks
Metadata
Owner
Metadata
This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.