Ji-Ha

> uninstall your pytorch first : !pip uninstall torch torchvision torchaudio -y > > and then reinstall it by going to the pytorch official website to get the command Thanks...

How do you use GGUF format instead of GPTQ? Can you use with LM Studio to host? It would be very great to run inference shared on CPU+GPU. Also, how...

Thanks I am excited to get this working properly.

Well it seems like I managed to get the model loaded using llama-cpp-python with the new code in my repository but now I hit this error and I am stuck....