13416157913 comments

Results 3 comments of

13416157913

Hello, why I fine-tuning Qwen1.5-1.8B-Base and test with CMMLU, the model answer repetition

> Thanks for your interest in LMFlow! I am wondering if you are using `text-only` format for dataset? Currently the recommended dataset type is `conversation` (https://optimalscale.github.io/LMFlow/examples/DATASETS.html#data-format), with a bunch of...

Hello, why I fine-tuning Qwen1.5-1.8B-Base and test with CMMLU, the model answer repetition

> > > Thanks for your interest in LMFlow! I am wondering if you are using `text-only` format for dataset? Currently the recommended dataset type is `conversation` ([optimalscale.github.io/LMFlow/examples/DATASETS.html#data-format](https://optimalscale.github.io/LMFlow/examples/DATASETS.html#data-format)), with a...

Hello, why I fine-tuning Qwen1.5-1.8B-Base and test with CMMLU, the model answer repetition

> I conjecture the problem mainly comes from the template part. Since Qwen1.5-1.8B-Chat used their own conversation templates, it is highly recommended to use the same template with conversation format...