Is model downloaded from hf supported

[MQPAT]

when using convert.py to convert model downloaded from hf, shows error "params.json" file not found. But in hf model hub, this file does not exitst for llama2.

Did I do something wrong?

[awni]

Which model are you trying to download and convert? Can you provide a link? In general we don't support arbitrary downloads from HF, but the models in the MLX community are pre-converted, so you don't need to run convert.py on them.

[justfinethanks]

while this is open - i'm curious as well about a custom model that was fine tuned on mistral-7b-instruct-v0.2 and merged to safetensors on its own hf repo. I've tried modifying the mistral convert and infer script - and mushing up a bit of the qwen pretrained script into it, but the best i can get thus far is the model repeating 'text text text text text' or similar over and over. any ideas? thank you!

[MQPAT]

I am using the model of meta/llama2-13b-chat-hf, my model files are downloaded from the web instead of using the hf automodel library. The files can be found in link:

https://huggingface.co/meta-llama/Llama-2-13b-chat-hf/tree/main

I think I have found the problem, the hf format does not contain params.json file, in the meta format, this file is there.

I know I can use the converted model from mlx, but since I want to finetune the model and deploy in the mac platform, so I need to convert my own fintuned model instead of the original llama2.

Thanks a lot.

Regards Pengfei.

Awni Hannun @.***> 于2023年12月24日周日 22:17写道：

Which model are you trying to download and convert? Can you provide a link?

Just FYI, the models in the MLX community are pre-converted so you don't need to run convert.py on them.

— Reply to this email directly, view it on GitHub https://github.com/ml-explore/mlx-examples/issues/185#issuecomment-1868527277, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBU4UXZJ5CYDOALJBZASPFDYLA2RNAVCNFSM6AAAAABBBM4ILOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGUZDOMRXG4 . You are receiving this because you authored the thread.Message ID: @.***>

[mzbac]

For most Llama-like models in HF format, if they use the original Rope and SentencePiece tokenizer (which has a tokenizer.model), they should be able to be converted by using python convert.py --torch-path <path_to_torch_model> --model-name tiny_llama.

[MQPAT]

For most Llama-like models in HF format, if they use the original Rope and SentencePiece tokenizer (which has a tokenizer.model), they should be able to be converted by using python convert.py --torch-path <path_to_torch_model> --model-name tiny_llama.

I see, thanks a lot. I will try this later on. Currently we are using mlc to run llama2 for our production. I think the native support from mac will eventually outperform other ways. BTW, running original llama on mac is the cheapest way at this moment. Looking forward to mlx teams' future works.