mlc-llm
mlc-llm copied to clipboard
mlc-ai
Universal LLM Deployment Engine with ML Compilation
**Problem** There was a problem with the json config file properly overriding the default conv template specified. **Solution** I have rearranged the logic (with as little code as possible to...
(mlc-chat-venv) amd00@asus00:~/llm_dev/mlc-llm$ rm -rf build (mlc-chat-venv) amd00@asus00:~/llm_dev/mlc-llm$ mkdir -p build && cd build (mlc-chat-venv) amd00@asus00:~/llm_dev/mlc-llm/build$ python3 ../cmake/gen_cmake_config.py Use CUDA? (y/n): y Use Vulkan? (y/n): n Use Metal (Apple M1/M2 GPU)...
This PR introduces MiniGPT4, a model that enables large language models to see. It is in the process of verification and performance optimization, and the progress will be shown below....
I can run the build in models after downloading them (vicuna-v1-7b-q4f16_0 and RedPajama-INCITE-Chat-3B-v1-q4f16_0). But I can't run vicuna-v1-7b-q3f16_0 after successfully downloading it which should be less demanding than the q4...
(gh_mlc-llm) amd00@asus00:~/llm_dev/mlc-llm$ ll build total 122328 drwxrwxr-x 5 amd00 amd00 4096 6月 12 22:31 ./ drwxrwxr-x 19 amd00 amd00 4096 6月 12 22:12 ../ -rw-rw-r-- 1 amd00 amd00 34377 6月...
## 🚀 Feature Support stable diffusion models! SnapFusion models can run in the phone within 2 seconds! [https://arxiv.org/pdf/2306.00980.pdf](url) ## Motivation Text-to-image diffusion models can create stunning images from natural language...
## ❓ General Questions When I completed the compilation and tried to run `./mlc_chat_cli --local-id ../../dist/vicuna-7b-all-v1.1-q3f16_0`, I got follows error: ` terminate called after throwing an instance of 'tvm::runtime::InternalError' what():...
## ❓ General Questions Hello. We have a LLM model. and we want to use mlc-llm solution. Our model is trasformer base and we customize structure. Can we use your...
Introduce microsoft/Phi-3 from https://huggingface.co/microsoft/Phi-3-mini-4k-instruct cc. @CharlieFRuan @MasterJH5574 @tqchen
