mlc-llm
mlc-llm copied to clipboard
mlc-ai
Universal LLM Deployment Engine with ML Compilation
## 🐛 Bug RWKV v6 models fail to compile with latest mlc_llm. Edit: Also it seems that there's currently only rwkv v5 compiling test in ci. Should rwkv v6 be...
## ❓ General Questions Hello friends, how can I use the parameter logits_processors to modify the current logit? Is there any interface on Android platform?
## 🐛 Bug  ## To Reproduce Steps to reproduce the behavior: 1. Download Command-R-Plus (either variant) 2. Attempt to quantize with convert_weight 3. Get above warning ## Expected behavior...
## 🐛 Bug Phi 3.5 mini crashes mobile app ## Expected behavior Phi 3.5 mini saying Initialize and then crashes the mobile app ## Environment - Platform: Android - Operating...
## 🐛 Bug ## To Reproduce Using this model [Phi-3-vision-128k-instruct](https://huggingface.co/microsoft/Phi-3-vision-128k-instruct) I got some bugs, need your help !!! For phi3-v problem, when I **converted model weight**, I got Error message...
[Bug] ValidationError: 1 validation error for MLCChatConfig conv_template Input should be an object
## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: I have followed the configuration and model files published by Siyuan on Hugging Face, but after running the local...
## 🐛 Bug 在下载后的模型点击对话时闪退，浮窗依然显示Initialize... 闪退的分析报告如下： 应用名称：MLCChat 应用版本：1.0 问题发生时间：2024-08-19 14:22:15 问题Trace： org.apache.tvm.Base$TVMError: ValueError: Check failed: (f != nullptr) is false: Cannot find function mlc.multi_gpu.SendFromLastGroupToWorker0 Stack trace: File "/Users/moyu/myworkspace/mlcllm/mlc-llm/cpp/serve/function_table.cc", line 139 at...
## ⚙️ Request New Models - Link to an existing implementation (e.g. Hugging Face/Github): https://huggingface.co/mosaicml/mpt-7b-instruct - Is this model architecture supported by MLC-LLM? (the list of [supported models](https://llm.mlc.ai/docs/prebuilt_models.html)) No. ##...
## 🐛 Bug Every model downloaded either complains download not complete, try to redownload, or when it is downloaded again or doesn't make that complaint, throws this error: MLCChat failed...
## 🐛 Bug Failed to generate resuults due to `RuntimeError: Check failed: input_shape[i] == heap_data[reg]` Steps to reproduce the behavior: I followed the official basic introduction tutorial and converted the...
