mlx-examples icon indicating copy to clipboard operation
mlx-examples copied to clipboard

Add -local flag to llms/hf_llm/convert.py for reading source HF models from filesystem.

Open kerekovskik opened this issue 1 year ago • 3 comments

This PR is to add a local flag (-l, --local) to convert.py to enable the reading of HF Models already on the filesystem during a conversion operation. Currently the script has to download the model weights from HF directly before beginning the conversion to MLX format.

Setting the --local flag disables uploads to huggingface during conversion as the hf_path argument would not specify the original model in the modelcard.

kerekovskik avatar Jan 09 '24 02:01 kerekovskik

I consolidated the fetch_from_local and fech_from_hub functions into one. Though I kept the fetch_from_hub function name as-is. Let me know if you'd like to change that to just load_model or something more neutral as it is not technically fetching from the hub if local=True is passed.

I also removed all the code related to non-safetensors.

kerekovskik avatar Jan 09 '24 15:01 kerekovskik

@awni , I think I've incorporated everything from your initial review. Please let me know if there is anything else for this PR.

kerekovskik avatar Jan 10 '24 14:01 kerekovskik

Thanks a ton! I will check it shortly!

awni avatar Jan 10 '24 15:01 awni