EISMANN-DEV

very slow GPU compared with CPU

15

Hi all ! model is working great ! i am trying to use my 8GB 4060TI with MODEL_ID = "TheBloke/vicuna-7B-v1.5-GPTQ" MODEL_BASENAME = "model.safetensors" I changed the GPU today, the previous...

How to use branches models ?

7

Hello everyone, Lets say i want to use - TheBloke/Llama-2-Coder-7B-GPTQ That model has a few branches such as main, gptq-4bit-32g-actorder_True, gptq-8bit--1g-actorder_True etc, if you scroll down. eg : https://huggingface.co/TheBloke/Llama-2-Coder-7B-GPTQ vs...