mlc-llm
mlc-llm copied to clipboard
mlc-ai
The sample installation code doesn't work on Macbook ARM
Hello there,
I found this problem while executing the sample code given for installation on Macbook M1. How should I resolve this?
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
Check failed: (lib_handle_ != nullptr) is false: Failed to load dynamic shared library /Users/dist/lib/vicuna-v1-7b_metal_float16.so dlopen(/Users/dist/lib/vicuna-v1-7b_metal_float16.so, 0x0005): tried: '/Users/dist/lib/vicuna-v1-7b_metal_float16.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
Stack trace:
[bt] (0) 1 libtvm_runtime.dylib 0x0000000108f16c98 tvm::runtime::Backtrace() + 24
[bt] (1) 2 libtvm_runtime.dylib 0x0000000108ee3929 tvm::runtime::detail::LogFatal::Entry::Finalize() + 89
[bt] (2) 3 libtvm_runtime.dylib 0x0000000108ee38c9 tvm::runtime::detail::LogFatal::~LogFatal() + 25
[bt] (3) 4 libtvm_runtime.dylib 0x0000000108ede159 tvm::runtime::detail::LogFatal::~LogFatal() + 9
[bt] (4) 5 libtvm_runtime.dylib 0x0000000108f04b2d tvm::runtime::DSOLibrary::Load(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 269
[bt] (5) 6 libtvm_runtime.dylib 0x0000000108f04d2f tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::$_0> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) + 175
[bt] (6) 7 libtvm_runtime.dylib 0x0000000108f1e9b6 tvm::runtime::Module::LoadFromFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 598
[bt] (7) 8 mlc_chat_cli 0x000000010062e4c2 main + 8402
[bt] (8) 9 dyld 0x000000020082051e start + 462
Looks like you are using x86 conda despite being on M1. You can perhaps try to use a conda that support arm64 natively for example, miniforge should come with M1 support
M1Max MacBook Pro running Ventura 13.3.1. Updated Conda (M1 version 23.3.1) and followed installation instructions. Works fine via cli.
Note that the cli looks for libraries in a hard-coded 'dist' directory, so important to follow the instructions closely.
Hello Team, Please make changes in the central strings by the json, I found this problem while executing the sample code given for installation on Macbook M1. How should I resolve this?
[bt] (0) 1 libtvm_runtime.dylib 0x0000000108f16c98 tvm::runtime::Backtrace() + 24
[bt] (1) 2 libtvm_runtime.dylib 0x0000000108ee3929 tvm::runtime::detail::LogFatal::Entry::Finalize() + 89
[bt] (2) 3 libtvm_runtime.dylib 0x0000000108ee38c9 tvm::runtime::detail::LogFatal::~LogFatal() + 25
[bt] (3) 4 libtvm_runtime.dylib 0x0000000108ede159 tvm::runtime::detail::LogFatal::~LogFatal() + 9
[bt] (4) 5 libtvm_runtime.dylib 0x0000000108f04b2d tvm::runtime::DSOLibrary::Load(std::__1::basic_string<char, std::__1::char_traits
@shashidhar771892 Hey thanks for sharing! What do you mean by central strings though? I'm happy to help but may need some extra info
I am having a similar error with RedPajama. Vicuna7b works fine though.
I have a Macbook Air, M1, Ventura 13.1, Conda 23.3.1.
This is the error trace:
(mlc-chat) MacBook-Air-2:mlc-chat ramoniglesias$ mlc_chat_cli --local-id RedPajama-INCITE-Chat-3B-v1-q4f16_0
Use MLC config: "/Users/ramoniglesias/development/mlc-chat/dist/prebuilt/mlc-chat-RedPajama-INCITE-Chat-3B-v1-q4f16_0/mlc-chat-config.json"
Use model weights: "/Users/ramoniglesias/development/mlc-chat/dist/prebuilt/mlc-chat-RedPajama-INCITE-Chat-3B-v1-q4f16_0/ndarray-cache.json"
Use model library: "/Users/ramoniglesias/development/mlc-chat/dist/prebuilt/lib/RedPajama-INCITE-Chat-3B-v1-q4f16_0-metal.so"
Loading model...
You can use the following special commands:
/help print the special commands
/exit quit the cli
/stats print out the latest stats (token/sec)
/reset restart a fresh chat
/reload [local_id] reload model "local_id" from disk, or reload the current model if `local_id` is not specified
[08:52:13] /Users/runner/work/package/package/tvm/src/runtime/dso_library.cc:125:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: (lib_handle_ != nullptr) is false: Failed to load dynamic shared library /Users/ramoniglesias/development/mlc-chat/dist/prebuilt/lib/RedPajama-INCITE-Chat-3B-v1-q4f16_0-metal.so dlopen(/Users/ramoniglesias/development/mlc-chat/dist/prebuilt/lib/RedPajama-INCITE-Chat-3B-v1-q4f16_0-metal.so, 0x0005): tried: '/Users/ramoniglesias/development/mlc-chat/dist/prebuilt/lib/RedPajama-INCITE-Chat-3B-v1-q4f16_0-metal.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/ramoniglesias/development/mlc-chat/dist/prebuilt/lib/RedPajama-INCITE-Chat-3B-v1-q4f16_0-metal.so' (no such file), '/Users/ramoniglesias/development/mlc-chat/dist/prebuilt/lib/RedPajama-INCITE-Chat-3B-v1-q4f16_0-metal.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
Stack trace:
[bt] (0) 1 libtvm_runtime.dylib 0x000000010d478808 tvm::runtime::Backtrace() + 24
[bt] (1) 2 libtvm_runtime.dylib 0x000000010d445499 tvm::runtime::detail::LogFatal::Entry::Finalize() + 89
[bt] (2) 3 libtvm_runtime.dylib 0x000000010d445439 tvm::runtime::detail::LogFatal::~LogFatal() + 25
[bt] (3) 4 libtvm_runtime.dylib 0x000000010d43fcc9 tvm::runtime::detail::LogFatal::~LogFatal() + 9
[bt] (4) 5 libtvm_runtime.dylib 0x000000010d46669d tvm::runtime::DSOLibrary::Load(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 269
[bt] (5) 6 libtvm_runtime.dylib 0x000000010d46689f tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::$_0> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) + 175
[bt] (6) 7 libtvm_runtime.dylib 0x000000010d480526 tvm::runtime::Module::LoadFromFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 598
[bt] (7) 8 mlc_chat_cli 0x0000000104be3d3e ChatModule::Reload(ModelPaths const&) + 478
[bt] (8) 9 mlc_chat_cli 0x0000000104be2ec4 Chat(ChatModule*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, int) + 212
@huevosabio this is likely due to you are using conda (x86) . Try to get a miniforge that works natively on arm64 instead