torchtune icon indicating copy to clipboard operation
torchtune copied to clipboard

Qwen 2.5 is here, Request for adding a model

Open vanshnawander opened this issue 1 year ago • 3 comments

The Qwen team has released qwen 2.5 base, coder, math models. They seem very promising. Requesting Team to add this model in torchtune.

### Tasks

vanshnawander avatar Sep 19 '24 17:09 vanshnawander

Hi @vanshnawander thanks for creating the issue. I took a quick look, seems like the architecture should be mostly unchanged. In that case we can reuse the model builders from Qwen2 here. It should be similar for the tokenizer but there are additional special tokens, along with minor changes to the chat template. I'm going to tag this as community help wanted since we'd love to have someone help out with this. cc @fyabc who added Qwen2 in case I'm missing any important details here.

ebsmothers avatar Sep 19 '24 18:09 ebsmothers

@ebsmothers Hi, thank you for you comments! Qwen2.5 only need to update tokenizer with new special tokens and chat template. I will take a look at this.

fyabc avatar Sep 23 '24 03:09 fyabc

As there is no updates here, I can take a look at it.

krammnic avatar Oct 11 '24 16:10 krammnic