Fails on M1 Mac: Cannot find vicuna-v1-7b lib in preferred path

[gagb]

I get the following error when I follow the instructions from the webpage for my M1 Mac

(mlc-chat) (....) MacBook-Pro local_chat % mlc_chat_cli Cannot find vicuna-v1-7b lib in preferred path "dist/vicuna-v1-7b/float16/vicuna-v1-7b_metal_float16.dylib" or other candidate paths%

Here is the output of ls dist

ls dist
lib		vicuna-v1-7b

ls dist/lib/
README.md                                 vicuna-v1-7b-q3f16_0-metal_x86_64.dylib*
mlc-chat.apk                              vicuna-v1-7b-q3f16_0-vulkan.dll*
vicuna-v1-7b-q3f16_0-metal.so*            vicuna-v1-7b-q3f16_0-vulkan.so*

ls dist/vicuna-v1-7b
float16		params		tokenizer.model

[tantanorange]

Similar problem. I followed the instruction here: https://mlc.ai/mlc-llm/#windows-linux-mac

When downloading the library from hugging face: https://huggingface.co/mlc-ai/demo-vicuna-v1-7b-int3/tree/main It does not contain the "float16" folder. Also, the script is looking for "dist/vicuna-v1-7b/float16/vicuna-v1-7b_metal_float16.dylib", which is not "vicuna-v1-7b-q3f16_0-metal_x86_64.dylib" in the "lib" main binary branch: https://github.com/mlc-ai/binary-mlc-llm-libs

Thank you so much for the amazing project! But following the current instruction does not seem to work now.

[ScorpionOO8]

Hi, I also get this error on an M1 Macbook Air:

Cannot find vicuna-v1-7b lib in preferred path "dist/vicuna-v1-7b/float16/vicuna-v1-7b_metal_float16.dylib" or other candidate path

Can the dev team please help us figure out what is going on? I am so excited to finally run LLM's on my M1's GPU for the first time!

[sudeepag]

Could you please try the suggestion in https://github.com/mlc-ai/mlc-llm/issues/149#issuecomment-1549801986 and ensure you have the latest package installed?

[philipqueen]

@sudeepag I'm having the same issues as described above on my m1 mac, and running the instructions in the comment from #149 gave me the following:

(mlc-chat) username@MacBook-Pro-2 ~ % conda list mlc-chat-nightly
# packages in environment at /Users/username/opt/anaconda3/envs/mlc-chat:
#
# Name                    Version                   Build  Channel
mlc-chat-nightly          0.1.dev34       34_g9129a4f_h1234567_0    mlc-ai

I'm not sure why 0.1.dev34 is downloading instead of 0.1.40

[tqchen]

The issue should be fixed by now, see also https://github.com/mlc-ai/mlc-llm/issues/149