Kanishka Dubey

> I am experiencing this issue as well with `python server.py --model llama-7b-4bit --wbits 4` > > The following did not resolve the issue for me. > > > I...

I am still facing an issue and getting the same error, here is the script I am running: $ python server.py --listen --wbits 4 --model MetaIX_GPT4-X-Alpaca-30B-4bit --gptq-for-llama --pre_layer 30 60...