mlc-llm
mlc-llm copied to clipboard
mlc-ai
python chat.py can not be run
This actually do not provide a loadable in huggingface repo, how does the tokenizer can load?
OSError: ./dist/models/vicuna-v1-7b does not appear to have a file named config.json. Checkout 'https://huggingface.co/./dist/models/vicuna-v1-7b/None' for available files.
What's more, does I also need tvm built with vulkan enable to run demo .so???
File "E:\codes\libs\relax\src\runtime\c_runtime_api.cc", line 131
TVMError:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: (allow_missing) is false: Device API vulkan is not enabled.
Hi @lucasjinreal , to build models on your own, you should clone the models' original huggingface repository, for example, for LLaMA, you should:
git clone https://huggingface.co/decapoda-research/llama-7b-hf.git dist/models/llama-7b-hf
The hugging face repositories under mlc-ai store compiled models, not original ones. The build script is only applicable to original models.
Regarding the second question, the answer is no if your compilation target is not vulkan.
@yzh119 As a matter of fact I don't want compiling my self, I just wanna load the lib you generated such as vulkan_x86.dll on windows, but apparently I can not even using this dll if no target tvm.device build against.
How can I just run the python with built-in dll on windows? (the prebuilt binary I can run sucessfully)
@hfyydd please specific which part need to change, to my opinion, this python code logically can not be run (except the plp who wrote this knows how to preparing) on any platform!!
