mlc-llm
mlc-llm copied to clipboard
mlc-ai
Universal LLM Deployment Engine with ML Compilation
Android MLCChat crashed on start --- error message: FATAL EXCEPTION: pool-2-thread-1 Process: ai.mlc.mlcchat, PID: 4697 java.lang.RuntimeException: java.io.FileNotFoundException: https://huggingface.co/mlc-ai/demo-vicuna-v1-7b-int4/resolve/main/float16/params_shard_7.bin at ai.mlc.mlcchat.Downloader.lambda$download$0$ai-mlc-mlcchat-Downloader(Downloader.java:98) --- apk file : https://github.com/mlc-ai/binary-mlc-llm-libs/raw/main/mlc-chat.apk
Similar to GPT4All, I just want to be able to import the library, then put an input into the code, run the code, and receive an output. Am I able...
## 📚 Documentation ### Bug (Broken link) - Link to the buggy documentation/tutorial: https://mlc.ai/mlc-llm/docs/tutorials/compile-models.html - Description of the bug: in the section marked “Run Build Script” there is a sentence...
## 🐛 Bug ## To Reproduce 1、 compile model **（It is OK！）** python3 build.py --hf-path databricks/dolly-v2-3b --quantization q3f16_0 2、compile mlc_chat_cli **（It is OK！）** cd build cmake .. make 3、run mlc_chat_cli...
## 🚀 Feature Allow to get only the new parts of a message ## Motivation I'm trying to stream data and currently backend.getMessage returns the full message: ``` fun getMessage():...
## ❓ General Questions I want to try it to build an iOS App, but still in research phase. Not sure if it's compatiable with flutter.
## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: ``` python3 -m mlc_chat.rest Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main mod_name, mod_spec, code =...
## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: 1. 1. 1. ## Expected behavior ## Environment - Platform (e.g. WebGPU/Vulkan/IOS/Android/CUDA): - Operating system (e.g. Ubuntu/Windows/MacOS/...): - Device...
Thank you for everyone! Finally, I success build. but i got new problem. Please help me. ## 🐛 Bug I restored "tvm_runtime.h" and success android build. But i got android...
## 🐛 Bug ``` ❯ python build.py --hf=eachadea/vicuna-7b-1.1 --target cuda Weights exist at dist/models/vicuna-7b-1.1, skipping download. Using path "dist/models/vicuna-7b-1.1" for model "vicuna-7b-1.1" Database paths: ['log_db/vicuna-v1-7b', 'log_db/rwkv-raven-1b5', 'log_db/redpajama-3b-q4f16', 'log_db/dolly-v2-3b', 'log_db/redpajama-3b-q4f32', 'log_db/rwkv-raven-7b',...
