text-generation-webui-colab icon indicating copy to clipboard operation
text-generation-webui-colab copied to clipboard

Published 20 hours ago • verified publisher icon camenduru

new to *free* google colab

Open nbollman opened this issue 1 year ago • 0 comments

Being on an NVIDIA T4, Is it possible to utilize xformers, and use exllamav2 as the loader for (mistral flavor of your choice)GPTQ 4bit 32gs ... I have a feeling it would perform blazingly fast with minimal degradation and great context... But you've spent more time on this...

nbollman avatar Oct 19 '23 01:10 nbollman