LLaMA-Factory icon indicating copy to clipboard operation
LLaMA-Factory copied to clipboard

Published 20 hours ago • verified publisher icon hiyouga

Add template&support(tested)

Open marko1616 opened this issue 10 months ago • 3 comments

What does this PR do?

Add support for #3184

Before submitting

marko1616 avatar Apr 12 '24 20:04 marko1616

Set param --use_fast_tokenizer true is in need

marko1616 avatar Apr 12 '24 22:04 marko1616

Template test pass using cli_demp.py image

marko1616 avatar Apr 12 '24 23:04 marko1616

Sft test pass(Lora)

marko1616 avatar Apr 12 '24 23:04 marko1616

难绷捏，似乎挺难完美复刻huggingface上说的完整的template

marko1616 avatar Apr 15 '24 12:04 marko1616

Some necessary changes have been made and this pr can be safely merged

hiyouga avatar Apr 15 '24 15:04 hiyouga

Set param --use_fast_tokenizer true is in need

fixed in e0dbac28450a0e1e0b84e1577ef785fc762c0b46

hiyouga avatar Apr 15 '24 15:04 hiyouga