mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

Published 20 hours ago • verified publisher icon EricLBuehler

Using uqff downloads the full safetensors weights.

Open chigkim opened this issue 4 months ago • 2 comments

Describe the bug

I ran the following, and it downloaded both uqff as well as full weights from Meta. I tried to skip -m, but it seems -m is required.

./mistralrs-server -i vision-plain -m meta-llama/Llama-3.2-11B-Vision-Instruct -a vllama --from-uqff EricB/Llama-3.2-11B-Vision-Instruct-UQFF/llama-3.2-11b-vision-hqq8.uqff

Latest commit or version

3e79d85

chigkim avatar Oct 05 '24 19:10 chigkim