LoRA icon indicating copy to clipboard operation
LoRA copied to clipboard

Question about the test set of the GLUE benchmark

Open James6Chou opened this issue 8 months ago • 1 comments

While reading the /examples/NLU/examples/text-classification/run_glue.py file, I noticed that the GLUE dataset only uses the validation set for generating results and does not measure accuracy on the evaluation set. Would it be better to evaluate the accuracy on the evaluation set using the model that performs best on the validation set in run_glue.py?

James6Chou avatar Nov 01 '23 14:11 James6Chou