mlc-llm

mlc-llm copied to clipboard

## 🐛 Bug Do not running Llama-3-8B-Instruct-q4f16_1-MLC ## To Reproduce Steps to reproduce the behavior: 1. conda create --name mlc-prebuilt python=3.11 2. conda activate mlc-prebuilt 3. conda install -c conda-forge...

BlindDeveloper

[Bug] mlc_llm generates randomly corrupted Unicode character when outputting Chinese

5

## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: 1. Install latest mlc-llm and mlc-ai in conda with python 3.12, running on an Apple Silicon (M1 Pro) MacBook...

LuRenJiasWorld

## ❓ General Questions I'm trying to add a model I'm interested in running with MLC-LLM, however is there a method for testing the intermediate `nn.Modules` that are part of...

mitchelldehaven

## ⚙️ Request New Models - Link to an existing implementation (e.g. Hugging Face/Github): ## Additional context

BlindDeveloper

[Question] Build on Android 32bit device 'armeabi-v7a'

7

## ❓ General Questions When checking the Android demo app and Build Android App from Source, only 'arm64-v8a' is supported. I wonder if it is possible to build the 'mlc_llm...

khkang80

[Model] Add use_qk_norm option for Cohere model

3

This pr updates `use_qk_norm` option for Cohere series models like Command-R-Plus.

tlopex

[Question] does mlc ai for android can run using nnapi?

2

Hello, i does android demo app possible using with nnapi support?

BlindDeveloper

[Bug] When I enable "<|im_end|>" as stop_str in qwen2 configuration, the final output seems to be truncated.

4

## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: 1.Do not set ``. Of course, my fine-tuned qwen2 model will output ``. But the problem is that it...

Moxoo

[Question] APK 安装后下载模型能手动下载到手机吗？

2

## ❓ General Questions hello，great pr 我安装apk后，点击下载模型，进度一直到不了100%，还在下载，下载V版本的下载60个bin模型了， 请问哪里有问题么？ 我想手动下载， 而且出现了模型重复下载的 情况，比如params_shard_0-1.bin 是重复下载吧

ucas010

[Bug] Qwen2-1.5B Q4F16_0 - libc++abi: terminating due to uncaught exception of type std::length_error: vector

1

## 🐛 Bug I believe when the final token completes or is about to be completed in a request, the entire app crashes with `libc++abi: terminating due to uncaught exception...

digisomni