LMDrive icon indicating copy to clipboard operation
LMDrive copied to clipboard

Published 20 hours ago • verified publisher icon opendilab

Questions about model.

Open qqqq-0 opened this issue 1 year ago • 1 comments

请问有试过放开llm的参数进行训练吗，有试过的话请问跟目前 frozen llm的比精度怎么样？

qqqq-0 avatar Feb 26 '24 02:02 qqqq-0

你好，有尝试过添加lora进行finuetune，training loss会低一些，但是实际closed-loop的性能没有测试过

deepcs233 avatar Mar 05 '24 12:03 deepcs233