mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

Published 20 hours ago • verified publisher icon EricLBuehler

Question about UQFF

Open schnapper79 opened this issue 4 months ago • 4 comments

I used your example to create a UQFF file from Lama-3.2-90B.

Now i am really struggling with loading the model from this UQFF-file. i try to use --from-uqff, but without adding -m and a model, it wont run. But if a add -m model it loads this model and safetensor files. So what's the trick?

schnapper79 avatar Oct 08 '24 20:10 schnapper79