transformerlab
I use Apple mlx lora tainer to fine-tune qwen3 8b. It will always output {{{{{{{{{{{{{{{{{{{ It seems that it does not match the end character.
I use Apple mlx lora tainer to fine-tune qwen3 8b. It will always output {{{{{{{{{{{{{{{{{{{ It seems that it does not match the end character.
Hi, It seems to me that since you wrote the template as {{messages}}, the adaptor somehow only learnt to predict the "{" character because that is the most prominent in each entry. Rather than this, if you could format your dataset into rows of system and user messages and provide a template after that on the new dataset, then the adaptor might train properly. We will still have a look and see if there is any other issues with the end character as you mentioned
why , So I have a chat dataset. How should I write it? How should I make a chat dataset?
The MLX LoRA trainer plugin takes a very simple approach and essentially doesn't handle any formatting for you. i.e. It does not convert your JSON into a chat template that the model was trained on. Instead it is just passing that JSON object to the model directly and teaching it to respond like that.
So your options are essentially:
- Use the Transformer Lab template editor (which uses Jinja) to apply a chat template.
- Pre-process your dataset to make 1 easier
- Depending on what your dataset looks like, maybe the chat template is not important since Qwen 3 is already trained well to chat. For example, I have previously had success just supplying questions and answers as raw unformatted text.
I realize this is not easy. We have an issue to hopefully improve this in the future: https://github.com/transformerlab/transformerlab-app/issues/168
Some additional conversation about the Qwen3 chat template to get an idea of what it looks like: https://www.reddit.com/r/LocalLLaMA/comments/1klltt4/the_qwen3_chat_template_is_still_bugged/
i use ' mlx_lm.lora --model /Users/caozhihui/ai_model/Qwen/Qwen3-8B --train --data ./data' try it again and see if there is no problem
I used mlx_lm.lora command to fine-tune , and then imported the fused model into transformerLab , and it output what I wanted, but when I asked the same question for the second time, it would output <|endoftext|>
Hmmm that is peculiar. <|endoftext> is the pad string on this model so it shouldn't display that. But what's weirder is that you didn't get a response before that.
It's hard for me to guess what's happening, especially since I can't read your chat unfortunately. Are you using a dataset that is publicly available? Perhaps I could look at it and try to see if I can spot the issue.
No! I am not sure when they added that, but I will look into this and fix! Thanks for finding this!