llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

Published 20 hours ago • verified publisher icon ggml-org

Ziya support

Open etyhh opened this issue 1 year ago • 1 comments

Ziya is a large-scale pre-trained model based on LLaMA with 13 billion parameters.It add Chinese token. And model vocab size is not equal to tokenizer.

Could anyone give some hint here？

etyhh avatar Jun 07 '23 06:06 etyhh

follow the readme and expand the tokenizer on your own works well for me.

wyhanz avatar Jun 08 '23 03:06 wyhanz

See my repo on huggingface: https://huggingface.co/thatname/Ziya-LLaMA-13B-v1-ggml

thatname avatar Jun 15 '23 14:06 thatname

problem solved,thanks

etyhh avatar Jun 19 '23 10:06 etyhh