Lion icon indicating copy to clipboard operation
Lion copied to clipboard

Is ther possible to distill Llama3 to Qwen2-1.5B and how to do

Open cxchen100 opened this issue 5 months ago • 0 comments

As a test case, I want to distill Llama3 to small llm:fo example Qwen2-1.5B if it is possible and if loss function should be changed?

thanks~

cxchen100 avatar Sep 07 '24 15:09 cxchen100