Arthur Aardvark

Edit: Forgot to mention this (below) is all after a sleepless night...this issue + written, I'm looking back at an untouched version and I'm not sure I'm describing all of...

MM yeah. Well, I don't know the ins-and-outs of MLC but I'd figured it would be beneficial to integrate Omni into its ecosystem. I still think that is true because...

Man, I can't believe this wasn't done by the MLX team months ago! Thank you for this, should come in handy. I was going to suggest Mixtral as #1 but...

So does AWQ work? For some reason I could've sworn it was a form of GPTQ so I figure there's a possibility. Wish I had an AWQ model downloaded to...

Broke my heart when I went to try Mistral_Large 2-bit EQAT (AutoGPTQ) on my M1 and only then saw no Mac support 😭. Wondering when might that come around? If...

Ever figure that out? Curious myself...

Totally forgot, I'm also having problems with the tokenizer library. I don't think they're related but could be totally wrong. I'm 95% sure this is built with each new iteration:...

Okay figured out the Tokenizers issue. But the OP's error still persists. I'm putting the computer's architecture as arm64 if that matters. I think I'd already mentioned it, but updated...

> @justinh-rahb yep, seems promising! Yeah MLC-LLM is the ship! https://github.com/open-webui/open-webui/issues/1270 Would love to plug-n-play with it