Jon Snow

Mistral 7B is dominating the local LLM scene right now and your software doesn't load it. I need your software to work with it... Can we please make your software...

Being on an NVIDIA T4, Is it possible to utilize xformers, and use exllamav2 as the loader for (mistral flavor of your choice)GPTQ 4bit 32gs ... I have a feeling...