mlc-llm

mlc-llm copied to clipboard

In this PR I have used the new spec for PagedKVCache for mlc-llm model definition. I have also removed the usage of the MLC-LLM PagedKVCache and relied on the TVM...

annanyapr

[Bug] Rope doesn't work for llama-3

1

## 🐛 Bug When I pass prompt longer than 8192, e.g. 10000 tokens the model returns gibberish ## To Reproduce Steps to reproduce the behavior: 1. Run server mlc_llm serve...

bene-ges

[Bug] gemma-2-27b-it-q4f16_1-MLC output the incorrect content.

4

## 🐛 Bug When I use the following model, it will repeatedly output "\\\\..." HF://mlc-ai/gemma-2-27b-it-q4f16_1-MLC ## To Reproduce Steps to reproduce the behavior: 1. In a conda environment, install mlc-llm:...

rankaiyx

## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: ## Expected behavior I'm trying to deploy a model on Android using the latest MLC-LLM build. However, I encountered...

jordanqi

## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: 1. 1. 1. I encountered the following error while trying to package the project for Android (aarch64-linux-android) using Rust,...

jordanqi

## 🐛 Bug ## To Reproduce Steps to reproduce the behavior: 1.在MLCChat目录下用Androidstudio打开该项目 2.点击gradle的下载按钮，此时全部下载完成，没有报错 3.点击绿色的三角形：run，开始报错 ## Expected behavior 开始报错： 1.> Task :app:compileDebugKotlin FAILED e: This version (1.5.11) of the Compose Compiler...

Myl-Ma

I indeed found multi nodes support in source code, but it lacks of example to show how to run multi nodes

shengxinhu

Update mlc_llm.rst correcting for mlc from mlc_llm

1

Fixed verify installation command line from mlc_llm to mlc (fails when attempting to use mlc_llm). Verified actual installed module is mlc as shown below. ``` (mlc-prebuilt) agrajag@tor-orin:/opt/miniconda3/envs/mlc-prebuilt/lib/python3.11/site-packages/mlc$ ls -la total...

agrajagco

In this PR I have refactored RandomGenerator to UniformRandomGenerator and PhiloxRandomGenerator.

annanyapr

## ⚙️ Request New Models - Link to an existing implementation (e.g. Hugging Face/Github): **[SmolDocling-256M-preview](https://huggingface.co/ds4sd/SmolDocling-256M-preview)** - Is this model architecture supported by MLC-LLM? (the list of [supported models](https://llm.mlc.ai/docs/prebuilt_models.html)) **No** (...

temsa