miles-du

Same issue here. I have tried conbinations of the following flags to load it at start --wbits 4 --groupsize 128 --pre_layer 32 --load-in-8bit --model TheBloke_stable-vicuna-13B-GPTQ - but no luck. 12GB...

I get the error only with "TheBloke" ausboss Wizard works very well. It works even with pre_layer as a startup argument. Cheers