kcramp858

Results 7 comments of kcramp858

This still happens on 1.3.0.. turning off # "format": "json" fixes it.

I get the same thing with pretty much every model I try -- just tried Vicuna-13b-cocktail-v1-4bit-128g Loaded up model_type=llama, 4bit, 128g.. Using latest gptq-for-llama and latest text-gen webui; Win 10,...

So, is it possible to have multiple GPTQ versions installed to allow selection? How are we supposed to figure out which GPTQ version works with which model?

I believe it must be related to #192 because I was using it just fine, then after restarting the service, I started to get the same error as #192 and...

--update-- I just fixed it as follows: docker-compose down git pull docker-compose up --build ... my issue was that I was trying to go to 127.0.0.1:3000 in the web browser....

I modify debugger.py and add: ` print(system) print(prompt) print(model)` that way it prints what it is sending to chatgpt; then I look through that and see what I can remove....