SafeAILab
How to train eagle3 and support qwen2
How can we train eagle3 and support qwen2? We want to implement it in our business, but we didn't find any code for training eagle3 in the project.
The cnets.py file here caters to eagle3 model training and cnets1.py for eagle1/2 model training. We can adjust our scripts accordingly.
The
cnets.pyfile here caters toeagle3model training andcnets1.pyforeagle1/2model training. We can adjust our scripts accordingly.
How to adapt the Qwen2 model? I didn't find the script for training Qwen2.
The
cnets.pyfile here caters toeagle3model training andcnets1.pyforeagle1/2model training. We can adjust our scripts accordingly.How to adapt the Qwen2 model? I didn't find the script for training Qwen2.
Yeah, so I think you need to change some parts of the cnets.py / cnets1.py file to adapt to Qwen2 by taking a closer look at the qwen2 modeling file.
The
cnets.pyfile here caters toeagle3model training andcnets1.pyforeagle1/2model training. We can adjust our scripts accordingly.How to adapt the Qwen2 model? I didn't find the script for training Qwen2.
Yeah, so I think you need to change some parts of the
cnets.py/cnets1.pyfile to adapt to Qwen2 by taking a closer look at the qwen2 modeling file.
Thank you for your reply
The
cnets.pyfile here caters toeagle3model training andcnets1.pyforeagle1/2model training. We can adjust our scripts accordingly.
If I want to train egale2 model for qwen2.5-7b-instruct, should I modify the cnets1.py or not ? I think that cnets1.py already supports qwen2 models beacause we can inference with the author provided eagle2 of qwen2-7b-instruct.
The
cnets.pyfile here caters toeagle3model training andcnets1.pyforeagle1/2model training. We can adjust our scripts accordingly.If I want to train egale2 model for qwen2.5-7b-instruct, should I modify the
cnets1.pyor not ? I think thatcnets1.pyalready supports qwen2 models beacause we can inference with the author provided eagle2 of qwen2-7b-instruct.
I guess the author only open-sourced the inference part, not the training part.
I think the training file is the same for both Eagle2 and Eagle3. The model file differs withcnets1.py for Eagle2 and cnets.py for Eagle3.
hey @skylee-01 , I wonder have you successfully trained the qwen2 model with EAGLE3, cause I'm also planning doing the same thing.
hey @skylee-01 , I wonder have you successfully trained the qwen2 model with EAGLE3, cause I'm also planning doing the same thing.
Good luck to you
hey @skylee-01 , I wonder have you successfully trained the qwen2 model with EAGLE3, cause I'm also planning doing the same thing.
Good luck to you OK, I'll try
The training code has been released.
@hongyanz Can you help update the cnets.py code to make it compatible with the Qwen model?
