LoRA icon indicating copy to clipboard operation
LoRA copied to clipboard

lora-dim == lora-r ?

Open ardand4708 opened this issue 3 months ago • 0 comments

Hi I am studing on LoRA and thanks for your work. I have a simple question, which is really confusing me. Dose the two hyper-parameters lora-dim of the GPT-2 model (in the generative task) and lora-r of the RoBERTa model (in the NLU task) serve the same role just for different tasks? Is it correct that i only changing the value of lora-dim to make the GPT model woring wtih different tank?

ardand4708 avatar Mar 27 '24 08:03 ardand4708