LoRA icon indicating copy to clipboard operation
LoRA copied to clipboard
verified publisher icon microsoft

Question about reproducing RoBERTa base Fine-tune

Open tlsdbfk opened this issue 2 years ago • 1 comments

I want to reproduce the performance of RoBERTa base Fine-tune. Modifying the apply_lora from True to False in the roberta_base_cola.sh code does not produce proper performance. What else should I do?

tlsdbfk avatar Sep 18 '23 11:09 tlsdbfk

We took these numbers from previous papers.

The LR is tuned for LoRA which is why simply turning LoRA off doesn't reproduce the numbers.

edwardjhu avatar Oct 29 '23 00:10 edwardjhu